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ABSTRACT 


A model for investigat'.lie; native research and development programs is 
formulated. Cost-constrained optimization methods are used in an expected value 
formulation for systems which have reached the concept development stage. For 
the remaining projects, those in basic research and exploratory development, decision 
rules for altering funding levels are suggested. The principal variables considered in 
the model are (1) the relative value of a system, a subjective value judgement, 

(2) the expected life of the system, (3) decision maker time preference, and (4) the 
cost of the system. A management information system for implementing the model 
is proposed which allows the user to focus on the trade-off implications of any of 
the alternatives available for modifying the budget of the R & D program. The 
specific decision problems the model addresses are those faced by the R&D planners 
of the U. S. Army. However, the model could also be applied to industrial 
research and development programs. No extensive knowledge of mathematics is 
required of the reader; however, explanations of various matehmatical concepts 
are discussed in an appendix. 
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L INTRODUCTION 


The information processing capacity of any one man is limited. Faced with 
potentially enormous quantities of data pertaining to a decision problem, the 
task of the decision maker is to select the significant information from the trivial 
and to identify an appropriate course of action. However, even the identification 
phase of this process is a difficult task when the problem is as large as that of 
budgeting the research and development program of the U. S. Army. Narrowing 
the scope of the problem to one of selecting a budget for a particular system under 
development makes the problem more manageable, but the impact of this selection 
on the program as a whole may not be readily visible. With limited resources a 
decision on a budget for one system necessarily affects all the others. The decision 
problem then becomes one of identifying what should be sacrificed if any system 
is to be increased. In an R & D program as large as that of the Army’s, the alterna¬ 
tives are almost countless and the time available for considering the problem is 
limited. 

The Computer Assisted Research and Development Budget Optimization Model 
(CARDB0M8) is a cost-constrained, value maximization model which considers 
the interactions of the budgets of all the projects in the R & D program. It adds 
insight to the question; “Based on the information gathered during the planning 
phase concerning the value and cost of each proposed project, how should X 
billion dollars be distributed among these projects?” 

For development programs which are at or beyond the concept formulation 
stage of development, the model uses optimization techniques for providing insight 
into this question. The variables considered in the model are: 
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1. The value of a system (the end product of a developmer plan); a function 
of tone, need, useful life, uncertainty and dollar costs. 

2. The cost of a system; a function of development, procurement and oper¬ 
ating costs, performance characteristics and operational readiness dates, 
time and uncertainty. 

The model also deals with basic research and exploratory development projects, 
i.e., those not directly associated with a program which has reached the concept 
development stage. Optimization methods are not used for these projects, but 
decision rules are suggested for investigating alternative budgets. 

Methods for quantifying the value of a system are discussed at length. Although 
the meaning of the terms “value” and “system” are dealt with more explicitly 
later, it should prove useful to introduce them here. An R & D system is the end 
product that is expected to result from one or more projects, e.g., the goal of a 
system’s development plan. In tl e hardware area the Cheyenne helicopter or the 
MBT 70 tank arc examples of systems. In a non-hardware area such as research in 
human performance, a system is the specific knowledge that is expected to be gained 
from one or more research projects. For example a system in the category of 
human performance might be termed “extending the endurance limits of the individual 
soldier,” consisting of a number of research or test projects. Systems are then 
categorized by general type, e.g., air mobility, missiles, human performance, etc. 

Within these categories experts are asked to use their own subjective value judgement 
in rating the relative importance they place on the systems ii: their category. Methods 
are then devised for transforming the interval scale of measurement scores of each 
category onto an int-’rval scale for the entire R & D program so that the value of a 
system in one category can be related to the value of a system in another. Thus 
value, rather than being some intrinsic property of a system, is c pc cptive notion 
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derived by considering what the system is designed to accomplish, and how much this 
capability is required. 

A. FORMULATION OF THE RESEARCH AND DEVELOPMENT PROGRAM 

Before beginning the development of the model, it should prove useful to 
briefly discuss how Program 6, Research. Development, Test and Evaluation (RDTE), 
of the Army budget is formulated.* 

Requirements for new weapon systems, and therefore R & D projects, are 
identified by a number of organizations. First, the intelligence community makes 
known the current and projected capabilities of potential enemies. Second, the 
studies conducted by the Combat Developments Command and other agencies con¬ 
cerning the organization and tactical employment of the Army uf the future identify 
areas in which weapons technology must be advanced. Additionally, technological 
advances have their own way of creatii " needs for new systems. For example, 
a breakthrough in research on tank engnes may lead to the decision that the most 
cost-effective step that can be taken is to develop a new tank. This ties in closely 
with a fourth way that ne ids arise, technological obsolescence. Some systems, 
designed with the technology of the past, have lost much of their initial value due 
to counter developments by potential enemies. 

All of these needs, or requirements, are channeled into the Army’s in-house 
R & D community for validation of the need and investigation of possible alter¬ 
natives. This process, which includes extensive investigation of the new technologies 
that would have to be developed to support the various alternatives, is called vhc 
concept formulation stage. Finally the point is reached where a development plan 

*For a more detailed discussion of the Army’s R & D organize n and procedures, 
as well as the overall Department of Defense management of K ic D, >ee Sanders [ref. 1). 
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item projects in Program 6 of the Five Year Defense Plan (FYDP), a planning and 
budgeting document maintained by the Army’s Chief of Research and Development 
(CRD). It should be noted at this point that not all of the more than 600 line item 
projects in Program 6 are directly related to a specific system. Many of them per¬ 
tain to overhead of the R & D in-house community (Sub-Program 6.5 — Management 
and Support) as well as basic research (Sub-Program 6.1) not directly associated 
with a particular system.'* Program 6, like all the Army budget programs, might 
be called a living document. As was mentioned, new projects are added every time 
a new DCP is approved. Additionally, within the bounds of the threshold points, 

DA may often alter the funding levels of various projects. However at some point 
in time during the annual budget cycle, the CRD submits the particular Program 6 
that he recommends for inclusion in the Army budget to be submitted to DOD for 
approval and forwarding to the President. For tne purposes of this paper, this five 
year funding schedule for all Program 6 projects and the funding schedules for each 
of them in the years beyond the FYDP up to the last year in which RDTE funds 
are planned will be called the base case . 

B. THE PROBLEM: INVESTIGATING ALTERNATIVE BUDGETS 

Once a Program 6 base case has been recommended by the Chief of Research 
and Development, the final decision on the program to be recommended to the 
President is far fro..i reached. The Army’s Budget Review Committee (BRC) and 
other top-level committees as the Secretary of the Army of the Chief or Staff may 
direct must then investigate how this base case fits in with other Army programs, 
and what adjustments might or must be made to insure compliance with national 

^More detail concerning the content, of Program 6 will be pres :ed in Section II, 

C, “A Taxonomy of R & D Systems.” 
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security objectives and the DOD fiscal guidance constraints. This is not to imply 
that these considerations are disregarded in arriving at the Program 6 base case. On 
the contrary, it was formulated throughout the year in an iterative fashion with just 
these considerations in mind. However, during the “budget crunch” months of 
October through December, difficult trade-off decisions must be made, and the BRC 
plays an increasingly important role.^ In essence they must investigate alternative 
mixes between Army programs which in turn determine the level , or total budgit 
constraint, of each. To do this with as much insight as possible in order to avoid 
setting arbitrary limits on the levels of programs, it is also necessary for them to 
investigate mixes within the programs. Unfortunately the time constraints of the 
PPBS cycle may limit this investigation substantially. It is not inconceivable to 
imagine that only the highest cost or critically important items get carefully investi¬ 
gated, while other projects in a program might have to experience something 
approaching an “across the board” percentage change, usually a reduction. While 
no empirical evidence is offered here to support this contention, it would appear 
logical that the more time available for identifying and investigating alternatives, 
the better the decision. 

Ultimately a budget gets approved by the Secretary of the Army for forwarding to DOD, 
but the budgeting problem is still far from resolved. DOD continues to n.ake Program Budget 
Decisions which are either accepted or rebutted by the Services. Additionally, either because 
the Office of Management and Budget has altered the budget constraint of DOD as a whole 
or because DOD wishes to investigate the alternative mixes between the Services, the 

Army may be directed to submit n new program constrained at some different level. 

The sometimes severe time constraints on these important deliberations and 

the need for vast quantities of readily available information pertaining to individual 

^Although the BRC is not the only agency making recommrn ions to the SA 
and COS concerning the budget, for simplicity this paper will refer ly to them as 
the principal advisors. 
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programs has led the Office of the Assistant Vice Chief of Staff, Army (AVCSA) 
the Army’s coordinating office for matters pertaining to the PPBS, to move toward 
automated management information systems. This paper is directed at assisting in 
this effort within the context of the existing decision problem and organizational 
structure. Sue specific decision problems representing the range of options for 
altering the Program 6 budget are addressed. These are: 

1. Should the funding of a system be reduced? 

2. Should the funding of a system be increased? 

3. Should the schedule of a system be slipped, freeing funds for a particular 
year? (Slipping is defined as delaying the start of a proposed development 
program, or cutting it off somewhere in mid-cycle with the intent of 
continuing it at a later date.) 

4. Should a new system just firishing the concept development stage be 
added to the program? 

5. Should a system be dropped from the program? 

6. Should funds be added to or taken away from the total RDTE budget? 

The extent of the sis. decision problems addressed by CARDBOMB shows that its 
potential users might be the planning staffs of the Director of the Army Budget, 
the BRC, the AVCSA or the CRD. 

C. CONSIDERATIONS IN MODELING RESEARCH AND DEVELOPMENT 

There are characteristics of R & D programs, both military and industrial, 
that make them particularly difficult to manage. A brief introduction to some of 
these problems is presented here so that the reader will be cognizant of them and 
follow more carefully how each of them are dealt with in the model. Chief among 
these difficulties is uncertainty. 

The uncertainty of the need for a particular system, sometime called scenario 
risk, is usually the most uncertain factor. The intelligence commun.iy might forecast 
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that by 1986 every Russian soldier wi'i be equipped with a man-packed jet propulsion 
unit capable of moving him above ground at 30 knots for great distances. There 
may, however, be a considerable question as to the validity of the forecast. Addition¬ 
ally there may be uncertainty as to the best approach to counter this move. Finally, 
no one can state for sure that by 1986 the Russians will be a potential threat. 

Perhaps, if the future were known, the need might be greater in countering a gamma 
ray gun development in Albania. 

The uncertainty of the availability date and ultimate performance characteristics 
of a weapon system undergoing development are termed technological uncertainty. 
Given the varying degrees of the state of the art in different technologies, it may 
become necessary to settle for less than the system’s performance objectives formu¬ 
lated in its current DCP. On the other hand a major technological breakthrough 
might allow the achievement of far more value than is currently envisioned. 

Similarly, the technological development problems might take so long to resolve 
that, by the system’s availability date, it has already been overtaken by events. 

For example, some counter R & D move by a potential enemy either now or in 
the future might seriously degrade the value of the system as currently perceived. 

This introduces the notion, to be explored in greater depth in Chapter 2, that the 
value of a completed system is not constant over time. 

The third significant factor is cost uncertainty. Particularly during the early 
concept formulation stage, R & D and other important cost estimates might be far 
off the mark. Even as development of the system continues, PEMA and OMA cost 
projections still may prove to be significantly different than projected. 

^Procurement of Equipment and Missiles, Army (PEMA) and Operation and 
Maintenance, Army (OMA) are two additional budget programs. Bo h the procure¬ 
ment and operation and maintenance costs are costs that must be s nsidered with 
RDTE funds in the "develop or do not develop” type decisions. 
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Examples of the implications of these uncertain factors illuminate their inter¬ 
dependence. If for a given system, RDTE costs have been estimated too low, PEMA 
funds for this system and others might have to be transferred, thus lowering the 
total, overall value to be gained. If the need has been overstated and Russia never 
fields jet-propelled soldiers, then the value of our jet-propelled Russian barrier may 
be zero, not to mention the value lost from other systems that might have been, 
developed with the same funds. 

Another difficulty encountered in the modeling of R & O is its lack of a unit of 
measure; i.e., it is meaningless to talk about dollars per unit of R & D. What type 
of an R & D unit of output could be defined such that two million dollars would 
buy twice as many units as one million? The MARK TWAIN, a Leontif Input-Output 
model currently being used by DA in studying the force structure budgeting problem, 
has as a necessary input the cost per unit for l\m,e X. The EXECUTIVE 
GUIDANCE/DECISION MODELS, an eutomated management information system 
also used in DA for studying the PEMA, OMA, manpower and force structure 
budgets, also require these linear cost estimating relationships. The activity analysis 
structure of these two models makes them inappropriate for R & D. primarily 
because of this lack of a unit of measure of output. 

It may be enlightening in this introduction to compare the purposes of these 
two models and the one proposed in this paper. The MARK TWAIN is primarily 
a costing model. The user inputs the force structure he is investigating and the 
model generates its cost. THE DEAN MACHINE, on the other hand, incorporates 
some built-in, parameterized decision rules for the alteration of the budgets of 

^The reader acquainted with Army budgeting models will pro! !y recognize 
this by its more familiar name, THE DEAN MACHINE, after its pr pal author, 

LTC A. M. R. Dean. 



specific line items in the budget program the user is investigating. For example, the 
purchase of a 214-ton truck might be the first priority for reduction in the PEMA 
budget, but only a five percent reduction can be absorbed. The user inputs the 
total budget constraint he is investigating for the PEMA program, and the model 
uses the decision rules to produce a new program constrained at this level. 
CARDBOMB has a similar objective. The user inputs a total RDTE budget constraint, 
and then based on one of a number of specific strategics he may wish to employ, 
a sirgle, mathematically optimal alternative is presented which both maximizes the 
value of the entire program and meets the total budget constraint specified. Of 
course as has been stated, “value” will be defined more explicitly later. 

Despite the problems posed by uncertainty, lack of a unit of measure and 
other factors, extensive efforts have been made to model R & D, both in the military 
and the industrial sectors. The primary emphasis, however, has been on project 
selection in the general field called capital budgeting. Project selection models have 
as their primary emphasis the decision problem faced when a new system is proposed. 
For the most part they are not concerned with incremental changes to the budgets 
of existing development programs. However some of these models could be very 
useful to the Army and are discussed further in the text. 

There are, however, at least two models which to some degree do address the 
problem of determining optimal incremental budget changes. The first of these 
was developed by McGlauchlin [ref. 4] for the Honeywell Corporation. In brief, 
division managers were surveyed and asked to score on a numerical scale how they 
rated the relative profit potential of the R & D projects that were on-going or 
proposed. The scores were aggregated to arrive at a number representing the value 
of a project relative to the value of one currently being marketed. iis gave the 


14 



planners a rough estimate of the expected profit potential of the projects rated. 

The scientific department was then asked what the earliest, latest and most likely 
completion times were for each project for various numbers of scientific teams 
assigned to it. All of these factors were then entered into a mathematical program 
which resulted in more efficient manpower utilization. 

This example points out a significant difference between modeling industrial 
arm military R & D programs (or any budget category, for that matter). The 
objective function in industrial models is generally uni-dimensional with one 
motive — maximize profit in dollars. In military R & D it is usually meaningless 
to assign dollar figures to the value of systems. A model called MEASURE I which 
was recently developed for DA by the Research Analysis Corporation (RAC) 
follows the same general formulation as the Honeywell model, but suggests a more 
appropriate ..ay of quantifying the value of jysiems [ref. 5). Many of the ideas 
that resulted in the model presented in this paper came from MEASURE I, and 
the author wishes to credit RAC for their work. However, the end product, 
CARDBOMB, is a model quite different from MEASURE I in its theoretical 
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D. QUANTIFYING THE VALUE OF A 
RESEARCH AND DEVELOPMENT SYSTEM 


Before a model of a complex program can be formulated, certain parameters 
have to be identified and measured. This chapter introduces the concept of the 
value of a system undergoing development; what it is and how it might be quantified. 

A. THE CHARACTERISTICS OF VALUE 

Terms such as value, utility, worth, effectiveness and cost can be quite vague 
if they are not carefully defined. This section introduces the meaning of value as 
used in this paper by first pointing out some of its characteristics. 

First, value may be either objective or subjective. Objective value reflects 
some generally accepted measure like dollars. Subjective value, often called 
utility, can only indirectly be put in a market context. The value of an item becomes 
the equivalent of what a particular observer is willing to give up to get it. Another 
observer might feel differently about the exchange. 

The values of two or more systems can also be commensurate or non-commensurate 
with each other. If the value of two items, each measured on different scales, can 
be transformed to the same scale, then they are said to be commensurate. For 
objective measures it is generally a simple matter to determine relations between 
value scales. If the value of item A is measured in dollars and item B in cents, 
one simply multiplies the value of B by 100 to make the two values commensurate. 

In the subjective measurement sense the value of two systems can be considered 
commensurate if the observer can relate the value of system A to that of B in terms 
meaningful to him. A pilot may say that as far as he is c oncerned. <> overall 
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value the Army will get from the Cheyenne program is greater than the value of 
the Cobra. Since be can compare these, they are commensurate to him. The value 
of two systems are non-commersurate in the subjective sense if the observer cannot 
relate their values. The same pilot may have no feeling for the value of the 
Cheyenne program relative to one of the tank programs because of his lack of 
familiarity with the latter. Even if he were thoroughly familiar with both, the 
differences in their missions may not allow him to reasonably compare their values. 

A third characteristic of value is concerned with how many attributes must 
be considered in its measurement. The value of a system is single attributed if it 
can be considered to accomplish only one significant objective, e.g., generate 
profit. It has multiple attributes if more than one significant objective must be 
considered in its determination. For example, it may move, shoot and communicate. 

Lastly, measures of value must be time dated and scenario related. The value 
of a system is generally not considered to be constant over time. It cannot be 
expected that the Cheyenne will have the same value in 1976 as it will in 1986. 
Similarly, a system’s value can be considerably different for two different scenarios, 
or operating environments, in which it might be placed. The value of the MBT 70 
tank may vary considerably depending on whether it is performing a combat mission 
in the Vietnam Delta or the Fulda Gap. 

Usually th„re is considerable disparity in the ease by which value can be 
measured for industrial projects and military systems. Risking a highly oversimplified 
generalization, it might be said that the measurements of value in business models 
are objective, singie attributed and commensurate, while in the military they must 
be subjective, multi-attributed, and most likely only subsets of systems are 
commensurate, i.e., Cheyenne and Cobra. 










B. OBSTACLES TO MEANINGFUL MEASURES OF VALUE 


If the value of a military R & D program is characterized in this way, significant 
obstacles are raised in arriving at meaningful measures. CARDBOMB uses various 
methods to handle each of the obstacles discussed in this section. 

If value is considered to be subjective, who is to be selected as the observer to 
render his judgement? Strictly theoretical models beg this issue and call him the 
decision maker. One might argue that this is the Secretary of the Army, but who 
can reasonably expect one man to have the necessary knowledge of each of the 
more than 600 projects, or even the time to think about it. Alternatively, some 
top level advisory board knowledgeable of all the systems might be surveyed, 
raising the additional problem of how all their replies should be aggregated, i.e., 
which one of them ha? the “right” answer. 

The model builder is also faced with problems caused by the multiple attribute 
nature of value. Which measures of effectiveness should be selected? How are 
they to be measured, weighted and aggregated to arrive at a single value measurement? 
What indirect objectives, that is, extraneous to the primary mission of the program 
should be considered? A glance at the following incomplete list will show that 
these indirect objectives seldom appear in any cost-effectiveness study, but have a 
definite impact on the decision problem. 

1. Support of an adequate R & D community. If the budget of System X 
is substantially reduced, forcing the bankruptcy of Company Y, how is 
the future R & D effort affected? 

2. Political. If System X is dropped, what will result from Senator Smith’s 
reaction? 

3. Enemy R & D reaction. If we develop System X, what can we expect 
the Russians to do? 
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Another obstacle to the development of a useful model of the RDTE Program 
is the non-commensurability characteristic of value. To construct a model, value 
must be measured. To construct an optimization model, the value of each system 
must be measured on the same scale, and the type of scale used is significant. This 
point should become clearer in the next section. 

Even within commensurate sublets of systems, i.e., those designed to accomplish 
generally similar objectives, additional obstacles arise tc complicate the measurement 
of value. Most systems are not indejjendent entities designed to operate in an 
environment all their own. More likely they achieve their value in a scenario in 
which they operate with other systems. Therefore the values of two or more systems 
may be interdependent rather than.independent. The value of Tank System A 
might be considerably different when Tank System B is developed and when it is not. 

Still more questions must be addressed in considering how to measure value. 
How should the three types of uncertainty be considered in the measurement of 
value? Should the value of a system be measured in the worst scenario envisioned, 
or the most likely? Should a conservative or optimistic view be taken in predicting 
the operating characteristics of the resulting system? And similarly, how should 
cost uncertainty be considered, since, as has been pointed out, an incorrect estimate 
may effect a loss in value to the program or other programs? 

This list of obstacles illustrates the magnitude of the problem of measuring 
value. All of these considerations must be addressed in determining how well this 
or any other R & D model represents the real world. Following the development 
of the valu"* measurement segment of thii paper, we will return to this list to 
discuss how each obstacle was handled. 
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C. A TAXONOMY OF RESEARCH AND DEVELOPMENT SYSTEMS 


In moving toward the goal of finding a method for making the value of as 
many RDTE projects commensurate as possible, a taxonomy, or classification 
scheme, must first be defined whereby “ reasonably commensurate subsets” of 
systems can be identified. A reasonably commensurate subset will be loosely 
defined as a group of systems aimed toward the accomplishment of generally 
similar missions or objectives. 

The most logical place to begin such a taxonomy is with the one currently 
used by the Army and defined in the Army Strategic Objectives Plan (ASOP). It 
identifies each RDTE line item project as belonging to one of the following 24 
categories. 

1. Air Mobility 

2 Air Defense 

3. Tank/Antitank 

4. Communications 

6. Surveillance, Target Acquisition, and Night Operations (STANO) 

6. Surface Mobility 

7. Indirect Fire 

8. Infantry Weapons 

9. Logistic Support 

10. Electronic Warfare 

11. Command and Control 

12. Chemical/Biological 

13. Nuclear 

14. Ballistic Missile Defense 

15. Personnel, Care, Protection and Survival 

16. Human Performance 

17. Counterinsurgency and Special Warfare 

18. Environmental Analysis of Military Operations 

19. Mapping 

20. Construction Methods 

21. Research for More than One of the Above 

22. Research Associated with None of the Above 

23. Testing 

24. Management and Other Support 

Categories 1 through 14 are called Hardware Categories, 15 thiongh 10 are Non- 
Hardware Categories, and 21 through 24, Support Categories. This paper will use 
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the same classification scheme with the following modifications. Eliminate category 
21 and assign its projects to the most valuable® system it supports. Remove 
category 23 and assign its projects to the category of the system it is designed to 
test. Eliminate category 22 and assign its projects to a new category called Basic 
Research. This category will have, in addition to those projects from the old category 
22, all basic research projects which are not a part of some development plan. 

For example, there may be a basic research project in support of the development 
of MBT 70. This project would be assigned to category 3, Tank/Antitank. Another 
basic research project may be aimed toward category 3 but not part of some 
development plan. It would be assigned to the new category, Basic Research, 

Most projects that are called basic research or exploratory development would fall 
into this new Category 22. To summarize, the categories for this paper are: 

Categoric: 1 through 20; no change. 

Categories 21 through 24; eliminated and replaced by, 

21 Management and Other Support 

22 Basic Research 

Now that a taxonomy has been defined, all that remains is to decide what 
belongs in each category. As was previously mentioned, the Army currently 
identifies each line item project as belonging to one of the categories. What is 
required for this model, however, is that systems be assigned rather than projects. 

Although this idea has been previously introduced, this last sentence requires 
considerable expansion. Up to this point the terms “project” and “system” have 
been used more or less interchangeably. In most of the Systems Analysis literature 
on capital budgeting, a project refers to that entity which results in value. In the 
RDTE budget, a project is usually one of a group of contracts, all of which go 

°Most valuable will be explicitly defined later. 
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into the development program which hopefully results in an operational system. 

In this model it is the value of the system which we seek to measure, not the 
projects. To avoid this ambiguity a system will be defined as one or more RDTE 
line item projects or portions of projects which, when taken together, expect to 
result in value to the Army. Thus the Cheyenne system from category 1 might 
be defined as consisting of three projects; advanced development, test and evaluation, 
and operational systems development.® This discussion pertaining to the assignment 
of systems to categories applies only to categories 1 through 20. Categories 21 
and 22 are treated differently than the others in the model. 

One additional classification will be assigned to each system. The system will 
be further identified as belonging to one of three time periods depending on when, 
under reasonable funding leveis, it can be expected to reach its operational readi¬ 
ness date (or, for Non-Hardware Categories, the objectives of the research will be 
reached). 1 ® The time periods are arbitrarily defined as: 

Tj : The five year period commencing with the fiscal year for which the 
budget is being formulated (hereafter called 1973). 

T 2 : The five years following Tj 

Tg : The five year period following the end of Tg. 

To summarize this classification scheme, with the exception of projects 
assigned to Categories 21 and 22, all other projects will be identified with a single 
system. Each system will in turn be assigned to a category and a time period. 


®The task of defining systems should be made easier by referring to the 
appropriate Development Concept Paper when one exists. 

1 ®A precise definition of how a system is time period classifii deferred 
until Chapter 3. 
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D. MEASURING THE VALUE OF SYSTEMS WITHIN A CATEGORY 


Having discussed the characteristics of value, we now turn to defining and 
measuring it. The value of an R & D system will be loosely defined as a notional 
(subjective) judgement made by an individual when he considers the capabilities 
of the system for meeting a specific need (or set of needs). Since this judgement is 
made in relation to the capabilities of other systems, it is often called relative 
value. 

Many methods have been devised for measuring these subjective value judge¬ 
ments.^^ In general they vary widely in the ease with which they can be applied. 

One of the easiest to apply in terms of the time required is a “ranking-rating” 
method. Briefly, a survey respondent is first asked to ordinally rank the systems 
in order of importance. After this has been accomplished he is asked to assign a 
number between zero and ten to the value of eacn system, where ten represents 
the value of the system deemed most important. The assumption is then made 
that these numbers represent an interval scale of measurement of the relative values 
of the systems.^ Due to the relative simplicity of this method, it will be the 
one described in this paper. However the model can be used with value parameters 
obtained from other methods, provided that they can be considered to be from an 
interval scale of measurement. 

Assume for the moment that, contrary to reality, the value of each of the 
systems within a category^ and time period has a single attribute and the attribute 

^Burington (ref. 6] discusses the meriU and shortcomings of some of the 
methods that have been applied in the R & D context. Fishburn [ref. 7] reviews 24 
different measurement methods that have appeared in the OR and Economic literature. 

*^The implications of this assumption are discussed in Appendix A. 

l^The discussion of the value of systems within a category wil ’xclude Category 
21, Management and Support, and Category 22, Biisic Research. 1 1 e are treated 
separately in Chapter 4. 
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is definable in some unit of measure, W. It could then be said that the system in 

T 

the category, a time period T system, has value during this period T of Vy * y W, 

where y is some non-negative real number. If this were an industrial model, W 

might be a dollar unit of measure and y the number of dollars, resulting in a 
T 

value such as Vy = S26 dollars. The fact that such a real numbered value like 
326 dollars cannot be measured for military R & D systems will be shown to be 
immaterial. 

Suppose that a group of high-level DA planners, all experts in the systems of 
the j^ 1 category, were surveyed and asked to ordinally rank the value of the time 
period T systems of the category considering the following factors: 

1. The performance characteristics of the resulting system will be exactly 
as specified in its current DCP. 

2. The system’s development costs are immaterial. 

3. The system’s operational readiness date will be as specified in the DCP. 

4. The system’s procurement and operating costs are a significant consideration. 

6. The threats these systems may have to face are also a significant 
consideration. 

Suppose further that, once an ordinal ranking was established, this same group 
of experts could respond to a second survey asking them to rate the value of each 
system on a scale of zero to ten, where the value of the moot important system is 
taken to be ten. Assuming the response to this survey establishes an interval scale 
of value, we then have a measure of the value of the i^ 1 system relative to the value 
of the system, where the system is the one deemed most important. 

Arbitrarily define this most important q^ 1 system as the time period numeraire.^ 

l^he numeraire system is that system to which all the others re measured 
relative to; the denominator of the ratios. 
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We then have for each of the 20 categories and 3 time periods, a set of real numbers 


between zero and one; 


v T - 

v iJ 

V T . 

v qj 


for 


i “ 1. • • •» 

T - 1, 2, 3 
j * 1 , . . . , 20 


where nj j ■ the number of time period T systems in the j** 1 category. For example, 
for 

j ■ 3 Tank/Antitank 

T = 1 Systems which are first available during the time frame 73-77. 
i * 2 Arbitrary assignment. Suppose it is “Antitank Gun Killer”, 
q ■ 1 M3T 70 (Assuming it was deemed most important) 


Then 


l&a- o.9 

' 1.3 


would imply that the consensus of the subjective value judgement of the group of 
Tank/Antitank experts determined that the value of having Antitank Gun Killer 
during the time frame 1973-77 was only 90 percent of the value of having the 
MBT 70 tank . 15 


Once these parameters are established it is then desirable to get a measure of 

how the value of each system is degraded in time periods subsequent to the system’s 

introduction into the inventory, considering the most likely technological obsoles- 

ence and enemy counter R & D moves. For example, the Tank/Antitank experts 

2 3 

might respond with time degradation factors of a^ 3 = 0.66 and a^ 3 =* 0 . 1 , 
implying that the value the Army will achieve from MBT 70 during the time 
frame 1978-82 is only two-thirds what it was in 1973-77, and during 1983-87, 

only one-tenth of what it was in 1973-77. 

15 How 1 consensus measure is reached is discussed later. 
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One further set of data is required to make the value measurements within the 


jtt* category commensurate. The systems within a category were partitioned into 
time periods to ease the problem of the survey respondents requirement to relate 
systems’ values being developed for different time frames. However they now must 
be surveyed to relate the values of the most important systems from each period, 
the time period numeraires. Let the time period 1 numeraire be the denominator 
in these two measurements (now called the category numeraire) and measure the 
values of T 2 and T 3 numeraires relative to its value in those time periods . That 
is measure; 




a 2 ,y ^ . 

a qj v qj 


and 


qj 


a 3 V L • 

qj qj 


for j = 1 , . . . , 20 


The simple formula for relating the values of all systems within the j category 


relative to its category numeraire is then. 





V T. 

qj 

a T . v i. 

qj qj 


for 


i = 1 , . . . , nj -p 

T = 1, 2, 3 
j = 1 . 20 


a l. = 1 

qj 1 


(i) 


Before continuing the development of the model it may be beneficial at this 
point to consider how such data might be obtained. One of the most widely 
researched techniques is the Delphi Method developed by Dalkey of the Rand 
Corporation [ref. 8 ]. It could be applied to this model in the following way. 

Write a carefully worded survey that included a precise list of the factors the 
respondents were to consider. Part of this list would be the performance character¬ 
istics and other information from the DCP of each of the systems in this category. 
Select at least ten respondents considered to be experts in the syst« s of this 
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category and issue them the survey individually.-^ Gather their responses and 
record how each of them rated the relative values and time degradation factors. 

Reissue the survey, but this time inclose the distribution of responses given for 
the last survey, and ask the respondents to reconsider their initial reply. Also 
ask that if they continue to rate a parameter outside the interquartile range of 
the group distribution of responses, to add a brief comment why they did so.*** 

Reissue the survey again, inclosing the new distribution of responses and all the 
anonymous comments, and ask for a final consideration. Dalkey has shown in 
a number of tests of the Delphi Method that considerable convergence in responses 
occurs after as few as three iterations of this controlled feedback technique. 
Unfortunately there is no way of knowing whether or not it is converging to the 
“true” parameter, some measure of the relation of the intrinsic values of the 
systems. 

Since the model will require a single measure to represent what is essentially a random 
variable parameter, this paper will consider the mean of the distribution of responses to be 
that measure, since in a statistical sense it represents the average of the responses. However, 
for the sensitivity analysis on the parameters discussed in Chapter 4, the end point of the 
interquartile range should also be recorded to provide the user with a feeling for the var¬ 
iance of opinion of his expert advisors. 

E. COMMENSURABILITY BETWEEN CATEGORIES 

The data collection effort outlined in the last section resulted in commensurate 
measures of value within each category, i.e., the value of the systems within the j“* 
category are measured on the interval scale of value of the category j numeraire 

1 fi 

AO In [ref. 8] Dalkey shows that the individual survey is preferred to a group 
meeting survey in that it eliminates a possible bias caused by one or more dominant 
personalities. 

17 

1 'The interquartile range is that interval containing the middle 30 percent of 
the responses 
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This method requires no mathematical assumptions, but presumes that each of the 


survey respondents are thoroughly familiar with the development programs of each 
of the category numeraire systems. 

The secondary method requires the assumption that the value parameters 
determined for the systems within a category be additive, i.e., 


v ij: v kj x.!*! 

V qJ V ci V qj 


This assumption means, for example, that if the values of systems i and K were 
rated at l A the value of the category numeraire, then the value of having both the 

IQ 

i and k systems is the same as the value of having the numeraire system. 

With the assumption of additivity of value the following simple equation can 
be formulated. 


r v- • • v? = N- 
A, »j i j 
1=1 


for j = 1, . . . , 20 


where Vj represents the "slack value” for category j, and Nj is a measure of 
aggregated needs. In words Equation 3 says that the needs in this category equal 
the sum of the values of all the systems being developed, minus some slack value. 
While it is recognized that "needs” are really multi-dimensional, (i.e., in category 
1 the Army has a need for both an attack helicopter and a surveillance helicopter) 
it is felt that the concept of aggregated needs is not an excessive abstraction since 


in the ASOP it is currently assigned a priority. Vj is included to cover two 
possibilities. First, due to technological uncertainty in the systems being 
developed, it is conceiveablo that as a hedge against this uncertainty more 
systems are being developed than are expected to be required. In this case 

^Further implications of this assumption, as well a - methods r testing its 
validity, are discussed in Appendix A. 
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Vj would be a positive quantity. Alternatively, intelligence on a recent R & D 
breakthrough by a potential eneav may have suddenly increased Nj, and sufficient 
time has not yet passed to reach the concept formulation stage for one or more 
new systems. Here V* would be a negative quantity. Obtaining a measure of V* 
might be accomplished in the following way. Again using the Delphi technique, 
survey the category experts with questions similar to, “Assuming the development 
of all systems proceeds as in their DCP’s, what system could be dropped from 
the program?” If the consensus were the system, it would imply that 

V qj V qj ' 

Alternatively, the consensus might be that no system can be dropped, implying 
that either the needs are exactly met or a gap between value and needs exists. 

The value of tnis negative slack might be determined by asking, “If in your opinion 
a system has to be added to the program to meet the needs in this category, to 
the value of which system currently in the program should the value of this new 
system be equal?” If the consensus were the l" 1 system, then 

v qj v qj ' 

A very useful equation can result from taking the ratio of Equation 3 for two 
different categories, in particular category 1 and category j, and applying some 
algebraic manipulations. 


pu-; * 

? v u • v ‘ N i 


Multiplying both sides of this equation by the quantity 






yields 


fL 

Nl 


qj 

^q»i 


v qJ 

V q»i 


3 

Ni 


'q,l 


Z v u - v* 

l_ 

I v u- v i’ 


(4b) 


and since 


V qj 


1/V 


q,i 


i/v f 


q,i ‘ ,T qj 

substituting for the second term on the right in Equation 4b we get 


_qj 

^q.i 


3 

Nl 


i 3 l 


1 v q,i V q»i 


z 3 i 


v i 


qj 


qj 


(4) 


Note that all the terms in the bracket in Equation 4 have already been 
determined. They are the values of the systems (or slack value) relative to their 
category numeriares. The term on the left, the value of the category j numeraire 
relative to die value of the program numeraire, is that quantity required for the use 
of equation 2. The term 


3 

Ni 


howev ir, is unknown. This represents the need during time period 1 for new 
category j systems relative to the need for Air Mobility systems. Determining 
these 19 parameters might also be done using the Delphi technique iih a group 
of high level DA managers. As was mentioned, it is currently dont .a the ASOP 
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on an ordinal scale for the first 14 Hardware Categories. The extension that this 
secondary approach would require would be to ordinally rank all 20 categories, 
and then determine 



in a manner similar to that used to find 



V qJ 


F. AN ALTERNATE APPROACH TO MEASURING VALUE 

In developing a useful, working model, trade-offs must be made between how 
much information the model can provide the user in his decision problem and how 
much effort is required in data collection. The approach outlined thus far already 
requires a considerable data collection effort, and even more is asked for in the 
next chapter. It should be pointed out that the parameters required to be obtained 
by survey, are for the most part, the same as those needed for RAC’s model, 
MEASURE I. This was purposely done to minimize the difference in the data 
collection effort. It should also be noted, however, that how the data are used 
to arrive at commensurate measures of relative value. 



is considerably different for the two models. This point is discussed further in the 
next section. 

Chapter 5 of this paper outlines a management information system that might 
be used to implement CARDBOMB. Mention is made that the user as the 
option to override any of the value parameters obtained thus far and insert his 
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own relative values; in essence, disagreeing with his advisors. However, the approach 
outlined in the last two sections gives him little insight into why the survey respondents 
replied as they did. The following approach might provide more insight, but at 
the cost of significantly increasing the data collection effort. 

Recall that the value of a system and the need for it are both multi-attributed. 
The primary approach requires a survey respondent to consider a vector of perfor¬ 
mance characteristics as well as a vector of possible scenarios, weight the importance 
of each characteristic and the likelihood of each scenario, and aggregate all these 
factors in his head to arrive at a single measure of relative value. Hence a great 
deal of information is not available to a user of the model should he wish to con¬ 
sider altering the value parameters of one or more of the systems. Some of this 
Insight might be regained by using the following alternate approach. Although 
it is outlined only for obtaining the relative values of systems within a category 
and time period, it could also apply to obtaining 




(depending on which method, the primary or secondary, was used). 

Survey the category j experts to obtain a list of what they consider the 
significant performance characteristics of the systems in the j 1 * 1 category to be. 
Label these k = 1, . . . , m. Follow this with a survey to determine the relative 
importance of each of these characteristics, relative, for example to k “ 1. Label 
these weighting factors as b^, where bj - 1. Again survey the category experts to 
determine the value of each of the systems relative to the value of the category 
numeraire for each of the characteristics. Label these 
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With the additivity assumption it could then by hypothesized that the value of the 
i^ system was equal to the sum of its value in achieving each performance 
characteristic, weighted by the importance of the characteristic; i.e., 


^ij Z v i,k - 

k-1 


We would then have: 



m 

v i,k 

v ij 

A 

b k ~~ 
q,k 

V qj 

m 

& 

. v q,k 
kv q,k 


“ v i,k 
b ' K Vk 

m 

Z *>k 

k=l 


for i = 1 ,. . . , n f rp 

J-l.20 


A simple example of this approach might help to clarify its use. Suppose the 

three surveys mentioned above for category 1, Air Mobility, resulted in the following 

set of data. Survey 1 determined that four important characteristics should be 

considered for Air Mobility systems. 

k = 1 Air speed 

k ■ 2 Maneuverability 

k *= 3 Weapons accuracy 

k ■ 4 PEMA and OMA costs 

Survey 2 determined that, considering the future needs for air mobility systems, 
the importance of each of these characteristics relative to air speed (determined 
to be most important) were 

bf ■ 1, b2 * 0.9, bg * 9.8, b 4 ■ 0.7 
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Survey 3 resulted in the determination that the value of the design specifications and 
costs for system 2 relative to the value of those for system 1, the category numeraire 
were, 


v l,l 


0.5, 





1.3, 


implying, for example that considering air speed alone, system 1 is twice as valuable 


as system 2. Using Equation 5 to aggregate this data we would have, 


V 2,l (1) (0.5) + (O.b) (1.2) + (0.8) (0.8) + (0.7) (1.3) _ 

---- 0.9 (to the 

v l,l 1 + 0 9 + 0 f * °- 7 nearest tenth) 


Because of the additional data collection effort that would be required, this 
approach is considered by the author to be inappropriate for this model. Discussion 
of it has been included in this paper primarily to indicate a limitation to the 
approach discussed in the previous two sections the limitation of the lack of 
insight provided the user in understanding the factors that were considered in 
arriving at the relative value and relative category needs parameters. This limitation 
might be partly overcome by having the survey respondents submit a brief 
paragraph on why they decided on the responses they gave. These might be 
summarized in a short paper for each system and referred to by the user when he 
is considering whether or not he agrees with a particular parameter. 


G. COMMENTS ON THE MEASUREMENT OF VALUE 

Section B of this chapter outlined obstacles to the effective measurement of 
value for R & D systems. It should be apparent that the value measurement 
portion of this model has dealt with these problems in various ways. In this 
section each of these obstacles is recalled in order to provide the re; !er with more 
information on which to judge the validity of the methods used. 
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Lack of a unit of measure. Recall that it was assumed that a unit of measure, 
W, existed, and that the value of each system could be measured as having some 
real number, y, of these units. On an interval scale of measure, W cancels out of 
the equation 

Vij_ 

V <h' *q W ' 

and the result is some dimensionless real number like 0.8. When a program 
numeraire (the denominator of all the ratios) is selected, it can then be said 
that the value of all the systems are measured on the value scale of this 
numeraire. For example 
V: 


'q.1 


« 0.8 


is equivalent to V; j = 0.8 V -i, implying that whatever real number is selected to 
represent V q j, V— must always be 0.8 of that number. 

The subjective nature of value. In any analytical study there is a natural aversion 
to using subjective value judgements as important parameters. Nonetheless, very 
few decision problems exist where they are not required to be made. Regardless 
of the pains to which a decision maker may go to collect “objective facts” bearing 
on the problem, he ultimately must subjectively weigh these facts when faced with 
a tradeoff decision. This model assists him by presenting him with a consensus 
measure of the subjective value judgements of his advisors, but allowing him to 
override their advice and use his own judgement',. 

The aggregation of subjective value judgements. Regardless of tne amount of 
objective information available to a group of experts, when they apply their own 
judgements it is unlikely that they will exactly agree. The Delphi M shod for 
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aggregating their advice is used in this model because it represents the extent of 


the state of the art in what might be called "opinion technology/’ To keep the 
model reasonably simple, a single measure of value, the mean of their responses, 
is used. However the decision maker has the right, indeed the need, to know the 
extent of the disagreement of his advisors. Hence the end points of the interquartile 
range of their distribution of responses will be recorded as a measure of the variance 
of their opinion. 

The non-commensurable nature of value. The model provides for partitioning 
the systems of the RDTE program, first into categories of systems which perform 
similar missions and then into time periods. After value judgements are made on 
these reasonably commensurate subsets, mathematically consistent methods are 
applied to relate the value of each system to the value scale of the program 
numeraire.^® It was stated earlier that these nm/uematical methods are considerably 
different in this model and MEASURE I. RAC’s model provides for the weighting 
of the relative value parameter within the j^ category by multiplying it by the 
relative need parameter of that category. In the notation of this paper this can be 
represented by 




N, 


This produces a different result than that obtained by using Equations 2 and 4, as 
the following simple example demonstrates. 

Assume a two category program with two systems per category where there 
exists absolute measures of value and need that are known with certainty as follows. 


^Besides identifying reasonably commensurate subsets, the pa ioning helps 
to eliminate bias. A proponent of Air Mobility does not rate the C enne 
relative to MET 70. 
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Category 1 


20 


Category 2 


V 2,l 

N 1 

V 1.2 

V 2,2 

n 2 


= 10 

■ 25, implying = 20 + 10 - 25 = 5 
= 15 


= 10 

= 20, implying = 15 + 10 - 20 = 5 


Although it is apparent that 



10 

20 


let us attempt to verify this using the methods of both models. Equation 4 
results in; 



20 

25 


20 ^ 10 _5 
20+20 20 
15 10 _5_ 

15 15 15 


15 

20 


and using this in Equation 2 yields; 

V 22 10 15 10 

-= — • — = — , m agreement with me data. 

V n 15 20 20 


The MEASURE I method would result in, 

V 22 V 22 N 2 10 20 8 

- *-• — = — • — = — , not in agreement with the data. 

V 11 V 12 N l 15 25 15 

This inconsistency is not the result of incorporating the concept of slack value. 

Even assuming Vj and are zero, implying = 30 and N 2 = 25, the RAC 


method results in 
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^22 = 10 25 = £ 10 
V 1X = 15 30 " 9 * 20 

The multi-attributed nature of valu e. This point was discussed at length in 
Section F where an alternate approach to measuring value was outlined. The 
principal approach requires the survey respondent to consider multiple attributes, 
but reduce them to a single measure of relative value. As mentioned, significant 
information may be lost to the user of the model when he desires to question the 
validity of a value parameter. 

Value as a function of time. The model requires that, the category experts 
estimate value degradation parameters reflecting the percent of the initial value 
remaining in time periods subsequent to the time period when the system is 
expected to be operational. Although five year time periods were defined, this 
is completely arbitrary. 

Scenario risk. This becomes a factor for consideration by the survey respondents, 
and thus is not made visible in the model. The respondent must consider how the 
systems would perform in various circumstances, and then apply his own judgement 
concerning the likelihood of these scenarios. Thus, if all other factors were equal 
if he considered a counterinsurgency war more likely than a major NATO encounter, 
the system designed for the former would have more value to him than one designed 
for the latter. 

Cost uncertainty. Except as this factor might be considered by a survey 
respondent, this important consideration is not included in the model. This 
applies also to indirect objectives, e.g., effects on the R & D community, political, 

^This discusr.ion is not to imply that this is the only difference in the value 
determination methods of the two models. 
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etc. Chapter 5 does outline, however, how sensitivity analysis con be conducted on 
cost parameters. 

Technological uncertainty. Recall that the value determination surveys 
directed the respondent to assume that performance characteristics and operational 
readiness dates would be exactly as projected in the current DCP’s. Thus value 
was measured independent of technological uncertainties. The next chapter 
introduces how the technological factors are considered. 

The interdependence of systems’ values. It would be unreasonable to require 
the Air Mobility experts to envision a future where- the Army operated with only 
one system from this category. Therefore the value parameters have a certain 
degree of interdependence. Yet the mathematical formulation of the model will 
require that these values be assumed independent. This inconsistency has important 
implications concerning the solutions the model generates, particularly with regard 
to the usefulness of the mode'; in providing insight into which systems might be 
dropped from the program. Discussion of these implications is deferred until 


Chapter 4. 







HI. RELATING VALUE TO COST 


The development of the model has thus far dealt solely with methods for 
determining commensurate measures of value for the RDTE systems. The reader 
may have already formed an opinion concerning the usefulness and validity of 
those methods. Should the concepts developed in the prev ious chapter be rejected, 
the remainder of the model can still be useful, providing that; (1) some method 
is used to arrive at commensurate measures of value of the systems as currently 
planned, and (2) some measure of how that value changes over time is determined. 

Recall that in the previous chapter, survey respondents were directed to 
disregard RDTE costs and technological uncertainty in determining the value 
parameters. We now turn to a consideration of these two factors in order to 
arrive at an intermediate goal — determining how the value a system is expected 
to achieve varies with RDTE dollar costs. 

A. SOME CONSIDERATIONS ON COST 

Costs are incurred in the implementation of the RDTE program for a number 
of resources, e.g., manpower, materiel, facilities and dollars. At some point in 
time any one of these resources might be a constraint on the program. However, 
since this is a budget model, it considers only the input of dollars. A still more 
restrictive definition of the costs considered must be used, however. To achieve 
the value determined in the last chapter, a system must be developed, produced 
and operated over its useful life. It would then appear obvious that the relevant 
cost considerations should be the total RDTE, PEMA and OMA cost ows. 



appropriately discounted.^ For models specifically pertaining to project selection, 
Le., those dealing only with the decision of which systems should be in the program, 
all of these dollar costs must be considered. However, since this model is designed 
to investigate RDTE budget alternatives, only RDTE dollar costs are considered 
directly. Recau, though, that survey respondents were directed to consider PEMA 

no 

and OMA costs in th' determination of value. To avoid ambiguity the term 
“cost” as used in the remainder of this paper will refer to the total RDTE dollar 
cost for a system unless otherwise specified. 

B. EXPECTED VALUE OVER TIME FOR A FIXED BUDGET 

What does an incremental change in the RDTE budget of a system buy in 
terms of value? Investigation of this question is an important step in determining 
how value varies with cost. 

First, if the performance characteristics (and thus, probably the value) are 
upgraded, an increase in cost is likely to result. Similarly increasing the budget 
may allow the value to increase, either from an upgrade in the performance character¬ 
istics or a reduction in uncertainty that the characteristics, as planned, will be 
achieved. 

A second way that value might vary with cost is in the time that is likely to 
be required in the development program before the system reaches operational 
readiness. An increase in the budget could possibly cause a shortening of the 
development time, while a reduction might require that the program be stretched. 

^While discounted costs are the appropriate consideration, discounting will 
be disregarded in tire remainder of the text. It could be easily incorporated into 
the model. 

^lt is shown later that considering only RDTE costs, rather th i total systems 
cost, will further restrict the use of this model in assisting with what might be called 
the ADD/DROP decision. 
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To determine how the value of each system varies with RDTE costs, it is 
necessary to gather more data from individuals intimately connected with the 
development of the system. To do this we might survey program managers with 
the following type questions. 

You are the program manager for System 127, Kickapoo helicopter. System 
127 is composed of the following projects with e base case RDTE funding schedule 
as shown. 


Project FY Budget (in Millions) 


Number 

Description 

73 

74 

75 

76 

77 

78 

79 

112 

Platform Engr Dev 

30 

30 

30 





163 

Platform Adv Dev 




50 

50 

50 


176 

Engine Engr Dev 

10 

10 






193 

Engine Adv Dev 



10 

10 




403 

Wpn Sys Dev 




20 

20 

20 


519 

Test and Eval 







5 


Total Annual Cost: 40 

40 

40 

80 

70 

70 

5 


Total RDTE Base Funding Cost: 315 


Question 1. What are the earliest, latest, and most likely operational readiness dates 
(ORD) at this base funding schedule? 

Question 2. Consider a total RDTE Funding level ten percent greater than base. 
How would you recommend using these funds to change performance characteristics 
and/or ORD?^ Record these new characteristics and the earliest, latest and most 
likely ORD. Also record the rew funding schedule. 


24 

Each major hardw;ire system has a program manager. For systems defined 
in Non-Hardware Categories, it is presumed that someone has the necessary famil¬ 
iarity to respond to such a survey. 

on 

Note that the program manager is asked to give his opinion f what might 
be called the technically feasible and optimal trade-off of performs ■ characteristics 
for ORD. It is presumed that his knowledge of the u chnologica! t >rtainty 
leads to a feasible trade-off. Whether or not it is the "best” tradeoff is a matter to 
be carefully itudied by users of the model. 
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Question 3. Answer question 2, but consider a total funding level 20 percent above 

base.^® 

Question 4 . At what total RDTE funding level would the Army be better off 
dropping this development program? 

Question 5. Letting Y be your answer to question 4, consider a funding level equal 
to Y + 1/3(345 - Y). Answer auestion 2 at this funding level. 

Question 6. Consider a funding level of Y + 2/3(345 • Y). Answer question 2 with 
the total budget constrained at this level. 

Note that question 4 identifies a “disaster funding level,” at or below which 
the program manager feels the Army is better off dropping the system. The other 
questions are designed to collect information on value changes and readiness date 
changes for five funding levels above disaster, two on the high and two on the low 
side of the base case funding level. 

Note also that for any of the four levels above disaster (excluding the base 
case) for which the program manager identified significant changes in performance 
characteristics, it would be necessary to treat the system as a new system and 
re-survey the category managers for different value parameters. 

Determining the earliest, most likely anu latest operational readiness dates is 
a technique from PERT theory used to construct a probability of completion 
distribution over time. The three parameters are usually used to construct a Beta 
probability distribution. For the purposes of this model, however, a simple linear 
cumulative distribution function will be defined as shovn in Figure 1, where P(t) 
represents the probability that the system is in an operational readiness state in 

^®The earliest ORD answer to this question defines to which t : c period the 
system belongs. 
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year t, and tg, t m and t p represent the optimistic, most likely and pessimistic 
completion times. 



Figure 1. CUMULATIVE PROBABILITY OF 

OPERATIONAL READINESS OVER TIME 

The one final set of data required for the model is indirectly a measure of 
its useful life. This data might be obtained by again surveying the category 
managers with a question l : ke, “what is the earliest, most likely and latest years 
you would expect this system to be retired from the operational inventory?” 

Since it is poss : that this probability distribution of retirement time is related 
to the operational readiness date land therefore to the funding level), this question 
should probably be asked for each binding levei tmd conditioned on the distribution 
of the operational roadir„ss date for that level. 

We now have reached the point where enough information is available to 
construct an expected value versus time curve for the i, system funded at 
level f, where: 

f = 1 Funding level Y, “disaster funding” 
f = 2 Y - 1/3 (Base - Y) 

f = 3 Y + 2/3 (Base - Y) 

f -- 4 Base case funding 
f = 5 Ten percent above base case 
f = 6 Twenty percent above base case 
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Figure 2 shows the typical shape of the curve for a time period 1 system. 



Figure 2. EXPECTED VALUE OF A SYSTEM OVER TIME 
FOR A FIXED FUNDING LEVEL "f" 


In Figure 2, the highest curve, encompassing both dashed and solid line segments, 
re H v~sents the value of a system when availability is not considered. It is constructed 
using a linear interpolation between the value degradation points determined in the 
category managers’ survey. The solid curve, the expected value of the system, is 
the product of the value of the system in year t times the probability the system is 
operational in year t. 

For each system a curve similar to Figure 2 would be defined tor each of 
funding levels 2 through 6. They may differ for each funding level in one of the 
following ways. 

1. If ih“ performance characteristics changed and resulted in a change in 
the value parameter for the system, the highest curve (value curve ) will 
be shifted up or down 


46 


















2. Different distributions of the probability of entry and exit times will 
undoubtedly occur, causing a shift of the expected value curve in the 
uncertain region (the years where the probability of availability is less 
than 1) to the right or left. 


C. THE TRANSFORMATION TO AN EXPECTED VALUE VERSUS COST CURVE 
Since a curve similar to Figure 2 can be constructed for each system i for each 
of five different funding levels f, a method is available for plotting how the total life 

expected value varies with total RDTE funds. Define the total life expected value of 

th 97 

the i system when funded at level f as follows. 

TVi ’ [E ti 3 EiVii f (e > 

where t" = max ft^ (the optimistic retirement year)J 
i 

For each system, apply Equation 6 for each of the five funding levels above disaster 
funding. A curve similar to Figure 3 will result. 



27 

For simplicity of notation the category subscript j will be drooped. The 
second subscript f will be used to denote which of the (i funding If - is being 
referred to. 
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An attempt might, be made to fit an analytical curve between the data points, 
but it is doubtful that much would be gained, and it would result in a difficult 
problem to solve for an optimum. 

Although Figure 3 shows total life expected value to be a simple function of 
total RDTE cost, actually it is quite complex. Incorporated in it is information 
concerning the system’s value relative to all others in the program, its expected 
useful life, technological uncertainty and all the other factors that have been con¬ 
sidered up to this point. 

Note might be taken of the shape of the curve. In the cost range [0, f = 6], 
or zero to 20 percent above base, it has a characteristic “S-shape”, reflecting no 
value achieved until a certain “buy-in” cost is paid. In the cost range [f = 1, f = 6], 
value is shown to be an increasing function with cost, although increasing at a 
diminishing rate. The curve is said to be conca'" 1 in the region [f = 1, f = 6]. Until 
such time as actual data is collected, there is no way of knowing for sure that such 
a curve will result for each of the systems. To assume that the curve is concave over 
the region of positive value is to accept the widely used assumption from economic 
theory of diminishing marginal returns. Simply stated this says that as funds are 
increased, a dollar buys more value, but not as much more as the dollar before it 
bought. 


^Appendix A lists more precisely how all the variables arc fi tionally 
related to TV-. 




IV, FORMULATING MATHEMATICAL PROGRAMS FOR 


THE SIX DECISION PROBLEMS 

Thus far the discussion has dealt primarily with defining the problem and 
gathering input data for the construction of total life expected value versus cost 
curves for each system in the program. In this chapter the discussion centers on 
how this information is used in mathematical programs to provide the decision 
maker with insight into selecting the “right” combination of the six options he 
has for altering the RDTE budget. 

A. ALTERNATIVE TIME PREFERENCE STRATEGIES 

Suppose international tensions had reached the state where it was increasingly 
apparent that World War III was likely to begin in the near future. Certainly the 
Army’s R & D effort would emphasize concentration on near-term systems to the 
detriment of those planned for operational readiness much further in the future. 
Conversely, in a world situation where war seemed highly unlikely in the near futore, 
the “best” strategy might be to concentrate the R & D effort on the later systems, 
gambling that the resources saved by a de-emphasis on near-term systems could be 
better spent on those being developed for the out years. Lacking certain knowledge 
of the future, there is no “best” strategy; nor should the am... t try to impose 
one. This is a strategy decision of great importance that should be made only at 
the highest levels. 

Recall that a time period 1 system was selected as the category numeraire for 
each category, and that systems from later time periods were related to it using 
the time degraded value of the numeraire. In essence, then, all systems are measured 
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in what might be called present value. This was purposely done to allow for the 
input of alternative strategies. Loosely define three different strategies in the 
following manner. 

Short range strategy : Relatively more emphasis is placed on achieving value in 
the near term. 

Neutral strategy : The importance of achieving value is equal in all time periods. 

Long range strategy: Relatively more emphasis is placed on achieving value in 
the out years. 

Any of ihese strategies can be incorporated ito the model in the following 
way. Ask the user to assign the number 1 to the time period he wishes to emphasize 
the most. Then assign fractional numbers to the other periods indicating their 
importance relative to the most important one. For example, parameters for a short 
range strategy might be (1, 3/4, 1/2), indicating that the user values operational 
systems twice as much in period 1 as in period 3. Similarly, parameters for a long 
range strategy might be (8/10, 9/10, 1), and for a neutral strategy (1,1, 1). 

Strategies can be incorporated into the model by using a time preference 
parameter, d^, defined as shown in Figure 4 (for a short range strategy). Note that 
a constant rate of change of time Dreverence is assumed, as well as a constant time 
preference for the years not between the data points. Once the user identifies 
tne three parameters reflecting the strategy he wants to investigate, u t is defined 
for each year t. The expected value curves are then modified to reflect the importance 
the user places on operational systems in any year by multiplying the expected 
value in yecr t times d t . The lesult is what is commonly called discounted 
present value. 
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d t 

1.0 

0.75 

0.5 

77 82 87 t 

Figure 4. TIME PREFERENCE PARAMETER 

The impact of inserting a time preference strategy can be seen in Figure 5. 

The solid curves represent the expected value as determined in the last chapter. 

They also represent a neutral (1, 1, 1) strategy. The dashed curves represent a 
short range (1, c/4, 1/2) strategy, while the dotted curves are a (1/2, 3/4, 1) long 
range strategy. Naturally, since strategies alter the expected value curves, the 
total life expected value versus cost curves will also be changed. 

Besides the obvious partiality for systems of a particular time period that 
results from selecting a strategy, strategies also cause changes in the relative import¬ 
ance of certain variables. For example, a short range strategy would lessen the 
advantage of a long life system, while a long range strategy lessens the importance 
of an earlier operational readiness date in a time period 1 system. All of these 

factors are incorporated into the model by the simple multiplication of d^ • E[Vj] , 

t 

B. DETERMINING THE OPTIMAL PROGRAM FOR A GIVEN STRATEGY 
AND BUDGET 

Once a strategy has been selected by the user, a specific TVj versus cost 
curve is identified for each system. Suppose the user wishes to use ( \RDBOMB 
to determine an optimal RDTE program for a given total 11 DTE bui.„et constraint 
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for FY 73 of B billion dollars.^® He desires that this be determined without con- 

on 

sidering dropping or slipping any of the systems currently in the base case program. 
Since dropping a system is not to be considered, attention can be directed to the 
TVj curves in the cost range above f = 1, the disaster funding level. That is, 
only funding levels 2 through 6 are to be considered feasible alternatives for each 
system. With the assumption that the values of the systems are independent of 
each other,the mathematical program can be formulated as follows:^ 

n 

Maximize: £ TVj(xj) 

i=l 

n 

Subject to: £ Cjf 73 ^ B 73 

where Xj = the total RDTE cost of the system 

iL 

Cj j t = the year t cost of the 1 system when funded at level f. 
Unfortunately (in a mathematical sense) the budget constraint is on the FY 73 
costs and not on the total RDTE costs. While it may be likely that the TVj(xj) 
curves are concave, it is possible that some of the TV- versus year t costs are not. 

As an illustration consider the two curves in Figure 6 . 

The right curve illustrates the case where year t costs as estimated by the 
program manager are higher at funding level 3 than they are at f - 4, certainly 
a possibility for some systems. Solving for an optimal solution when the curves 


^3 billion would represent the total funds allocated to RDTE minus the 
amount required for the categories of Basic Research and Management and Support. 
These are still being excluded from the discussion. Although the fir t year is being 
used as the budget year under investigation, the model can also be used to investi¬ 
gate constraints in years other than this first year. Discussion of this point is 
deferred until Chapter 5. 

^^These two decision options ore discussed in Sections D and 

3lThe implications of this assumption are discussed in Settle of this chapter 
and in Appendix A. 
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in the objective function are not concave can be a difficult task involving complex 
search procedures. However the following simple algorithm (solution method) will 
lead to an approximation to the optimal solution.^ 

Algorithm 1 

1. Calculate and label the slopes of the four linear segments of the TVj versus 
total RDTE cost curve above funding level f = 2 as (i, s), for i = 1, . . . , n 
systems and s = 1, . . . , 4 slopes. These slopes are measures of marginal 
value between the funding levels. 

2. Order these 4 n slopes in decreasing sequence, K = 1, . . . , 4n. 

3. Calculate the initial value of a running sum 

n 

R o = £ C i,2,73 : 
i=l 

the tot?.! FY 73 cost of the program when all systems are funded at 
f = 2. Set K = 1. 

4. Identify the i th system belonging to slope K. Calculate Qj^, the increase 
in FY 73 costs for this system when it is funded one level higher. 

% = C i,f+1,73 ’ C i,f,73‘ 

5. If Rft.i + Qk > B, go to step 8. 

6. If R K1 + Qj£ <L B set Rj£ = Rj^-j + Q& anc * ra * se the funding level of 
the i** 1 system to f = f + 1. 

7. Set K = K + 1 and return to Step 4. 

8. Stop. An approximation to the optimum has been reache 
32 

•‘How close this approximation is to the optimum is disclose. Appendix A, 
as well as more elaborate procedures for finding the actual optimum. 
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A simple example of this algorithm may help to clarify its use. Suppose the 
RDTE program consisted of two systems whose total value versus cost curves were 
as shown below. Additionally, the program managers for these two systems had 
estimated the total FY 73 funding for each of funding levels 2 through 6 to be*. 

Funding Level 

i 

P_ 3 4 5 6 

e . 1 f25 30 35 40 45 

System 2 60 70 80 90 100 




The slope ordering list called for in step 2 of the algorithm is reflected in the 
third column of the “Budget Incrementing Matrix” shown in Figure 7. Note 
Rq * 25 + 60 = 85. If the budget constraint being investigated were B = 115, 
the algorithm would stop at K = 4, reflecting both systems funded at their base 
funding levels. The total value accruing to the program would be 20 + 19 = 39. 



System 

f 

Marginal 

Value 

FY 73 
Funds 
Added 

R 

K = 1 

1 

3 

0.7 

30-25 

90 

2 

2 

3 

0.6 

70-60 

100 

.3 

1 

4 

0.5 „ 

35-30 

105 

4 

2 

4 

C.4 

80-70 

115 

- 5 

1 

5 

0.3 

40-35 

120 

6 

2 

5 

0.2 

90-80 

130 

7 

1 

6 

0.15 

45-40 

135 

8 

2 

6 

0.1 

100-90 

145 


Figure 7. BUDGET INCREMENTING MATRIX 
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As has been mentioned, once a strategy has been selected a TVj versus cost 
curve is identified for each system. Additionally, a budget incrementing matrix 
is defined. If a user wished to investigate some alternative RDTE budget level 
for FY 73, the recommended changes from the base case are clearly visib'e in 
the Budget Incrementing Matrix. For example if he wished to investigate a con¬ 
straint of B * 135, the model would recommend he consider raising system 1 to 
t » 6 and system 2 to f e 5. 

Studying Figure 7 should make the need for the assumption of concave curves 
apparent. If at some point, diminishing marginal value was not the case for a 
particular system, then the slope sequence list might, for example, call for an 
increase to f = 5 before f = 4 was achieved. For systems that might exhibit a 
“violation” to the law of diminishing marginal value, the following procedure can 
be applied as a modification to Algorithm 1. Consider a system whole TVj versus 
cost curve is as shown in Figure 8. For some reason little increase in value is 



Figure 8. NON-CONCAVE VALUE CURVE 

achieved in going from f s 3 to f ** 4 (marginal value near zero). The dotted line 
between f ■ 3 and f * 5 does represent the marginal value between t' ese points, 
however. The approximation to the optimal solution could be oht. d by using 
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the slope of this interpolated line in Algorithm 1. If it resulted in f = 2, 3 or 6 
being recommended for this system, the non-concave portion of the curve had no 
impact. If f = f> were selected, however, it would be necessary to check whether 
or not reducing this system to f = 4 would free funds to be spent on another 
system which would result in greater program value. 

0. TESTING THE SOLUTION AGAINST THE BASE CASE PROGRAM 

Suppose Algorithm 1 were used with an FY 73 budget constraint equal to that 
planned for the base case program. Undoubtedly a solution other than the base 
case would result, i.e., some systems would be funded higher than base funding 
(f = 4) and some would be lower. This could result from -my of the following 
reasons. 

1. The time preference parameter use. ' did not adequately reflect the 
actual time preferences of the collective leadership of DA. 

2. Some of the value, need, useful life or other parameters, all consensus 
measures determined by advisors or program managers, do not reflect 
the views of the decision makers. 

3. The base case program contains inefficiencies, i.e., after more careful 
analysis, a better program might result if funds were taken from some 
systems and given to others. 

4. The approximation to the optimal solution determined by Alogrithm 1 
was not close to opth .." 1 

5. Some combination of two or more of the above reasons. 

Resolving the differences between the solution as determined by the algorithm 
and the the actual base case program would be a difficult but useful exercise. 

First, it might identify inefficiencies in the actual base case, suggesting a better 
distribution of funds. Second, if the model is to be of any use at all in investigating 
alternative budget constraints, the parameters need to reflect a gene 1 agreement 
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on their validity. An investigation of the divergence of the solution and the base 
case might take the following form. 

First an attempt to resolve the time preference parameters to see if some 
particular set comes closest to producing a so’ution similar tc the base case. For 
example, starting with .1 neutral strategy (1, 1, 1), move increasingly in the direction 

OO 

of a short rang* strategy. 00 The criteria for a set of strategy parameters being more 
suitable than another could be tine number of systems funded at levels other than 
base funding. As an example, suppose this investigation resulted in the following. 

Time Preference Number of Systems 

Parameters Not at f = 4, Base Funding Level 

(1, 1, 1) 25 

(1, .95, .9) 20 

(1, .9, .85) 15 

(1, .85, .80) 5 

(1, .8, .7) 10 

(1, .7, .6) 20 

It might then be assumed that (1, 0.85, 0.8) most closely reflect the time preference 
of DA decision makers. Of course there is no way to prove that these parameters 
are “correct”, but they do imply that DA decision makers have acted as if these 
were their time preferences. 

The investigation might then continue with a careful analysis of the systems 
not at funding level 4. This investigation might result in agreement that the responses 
to the surveys were unrealistic in some cases and some of the input parameters need 
adjusting. Alternatively it might be agreed that the base case would indeed be 
better if some redistribution of funds were made. 

^Due to the nature of national defense, defense decision makers are generally 
presumed to prefer forces in being to systems planned for further in the future. 
Starting the investigation from a neutral strategy and moving toward strategies 
which are more and more short range should mit.imi/.e tne number < sets of 
parameters required for this investigation. 
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D. IDENTIFYING SYSTEMS TO BE SLIPPED 


The last two sections dealt with the problem of determining an optimal RDTE 
program for a given fiscal year budget constraint. The method formulated assumed 
that the user did not wish to consider slipping a system’s development schedule 
one or more years. As has been stated, slipping consists of delaying the start of 
a development program, or interrupting one that has already begun (such as between 
the engineering development and advanced development stages with the intent of 
starting it up again at a later date). This, of course, is a decision option open to 
the Army for altering the RDTE budget program for rny given year. We now 
consider how the model can assist the user in identifying systems which might be 
slipped one or more years. 

The first consideration that must be addressed is whicu systems might 
reasonably be considered for slipping. Certainly if one or more of the projects 
of a system scheduled for FY 73 are “carry-overs” from FY 72, it might be 
unreasonable to expect that it would be economical to interrupt work on those 
projects for one or more years and then begin them again. This fact alone will 
undoubtedly eliminate many systems from consideration for slipping. 

One must also consider what might be gained or lost from slipping the 
schedules of systems. The first “gain” might be called a reduction in technological 
uncertainty. The development cycles of systems follow a general sequential pattern 
starting with the concept development stag'' and progressing through engineering 
development, advanced development, test and evaluation, and finally, production 
and operational systems development. Between each of these stages of development 
a detailed review of the progress that has been achieved must be mmV and a 
decision issued on whether or not to begin the next stage. It might ■* determined 
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that, in order to overcome technical problems which have not been resolved, the 
best course of action is to slip the next stage one or more years. The decision 
problem associated with slipping a system’s development schedule to overcome 
technical difficulties is not addressed in this model. It is assumed that these decisions 
have already been made in arriving at the base case program. 

Besides reducing technological uncertainty, the Army might also wish to 
consider slipping the development schedule of one or more systems to save funds 
in a particular year. The model can assist with this decision option by identifying 
a measure of the value lost per dollar saved when a system is slipped. 

Suppose that for a particular time strategy and total budget constraint, 
Algorithm 1 resulted in funding level f being optimal for the i^ system. The 
expected value versus time curve for this system at this funding level might be as 
shown by the solid line in Figure 9. Under the assumptions that (1) distribution of 


EIVsJ, 



Figure 9. LOSS IN VALUE, L ( , CAUSED BY A ONE YEAR SLIP 

the probability that a system is operational by year t is affected by a one-year 
schedule slip only to the extent that it is displaced one year, and (2) the probability 
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distribution of the system retirement date is not affected by a one-year slip,^ then 
the loss in value, Lj, caused by a one-year slip to the development schedule of the 
i ^ 1 system can be approximated by; 


Em ift + ElV] it+ i 

L . —- a - < 7 > 

Dividing Lj by the FY 73 budget for the i " 1 system Cj f 73, would then determine 
an approximation to the loss in value per dollar saved by a one-ycar slip. This 
presumes that no money would be required to keep the program "alive” during the 
slip. If this were not the case, Cj f 73 would represent the difference. 

If a user of the model wished to consider slipping as an alternative for 
investigating different RDTE programs, this could be incorporated in the following 
way. 


Algorithm 2 


1. Use Algorithm 1 to determine the approximation to the optimal RDTE 
funding level f for each system when slipping is not considered. 

2. For the subset of systems identified as candidates for slipping, calculate 


1 r 
C i,f,73 

the loss in value per FY 73 dollar saved. 

3. Order the set of in increasing sequence k a 1, . . . , n. (The system 

with the smallest loss in value per dollar saved is first in this sequence, 
k = 1). Set k - 1. 

4. Identify the k ^ 1 system as the next system in the budget incrementing 
matrix to be increased if more FY 73 funds were available. Calculate 


^An alternative to these assumptions would be to gather more lata on how 
a one year slip would affect the earliest, latest, and most likely oper, ional readiness 
dates and system retirement dates. 
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TV k ,f+l • TV k,f 

C k,f+1,73 ' C k,f,73 


the gain in total value to the k^ system per FY 73 dollar added. 

5. I* Sj jj — (if the loss in total value caused by slipping system i one 
year’is greater than the gain in total value to system k when the money 
saved by the slip is transfered to system k), go to step 8. 


6. If Sj k < G^, slip system i one year and transfer the savings to system k. 

7. Let k = k + 1 and return to Step 4. 

8. Stop. Slipping any more systems and transferring the funds to other 
systems will only cause a net decrease in the total value of the RDTE 
program. 


E. IDENTIFYING WHICH SYSTEMS MIGHT BE DROPPED 

Another alternative for altering the RDTE program is to drop one or more 
systems. As has been mentioned, this decision problem is normally addressed on 
a system by system basis during the extensive progi - s review following completion 
of a stage of the development cycle, or when some threshold parameter of the DCP 
has been violated. It could also be addressed on a total program basis, i.e., given 
a particular FY 73 budget constraint, would more total value accrue to the RDTE 
program if one or more systems were dropped and the savings transferred to other 
systems? It has previously been stated that CARDBOMB can only indirectly assist 
in this decision problem for reasons which can be summarized as follows. 

The interdependence of the value parameters. As has been mentioned, the 
relative value of the i^ 1 system is a consensus measure of a group of experts who 
were not asked to envision a future where the i^ 1 system was the only one in the 
inventory. In other words, the value of this system depends on the value which 
results in other systems. The optimization procedure discussed thu- r has ignored 
this interdependence and assumed that the value achieved by one system does not 


62 




alter the value of another. This independence of value assumption says, for example, 
that the change in expected operational readiness date and performance characteristics 
caused by a change in the funding level of the i^ system in no way affects the value 
of other systems. For changes in value in the region around the base case funding 
level, i.e., plus or minus 20 percent of base funding, this assumption is probably not 
a bad approximation for most systems. This situation will undoubtedly not be the 
case when dropping a system is considered. While moving up the operational 
readiness date of Tank System A one year probably does not significantly affect 
the value of Tank System B, dropping System A would undoubtedly have a 
significant impact on the value of System B. Models have been developed which 
take into consideration the interdependence of value.In general they would 
require a far greater data collection effort and complex dynamic programming 
solution methods. 

The interdependence of value and total system’s cost. The surveys to determine 
the value parameters asked the respondents to consider PEMA and OMA costs in 
arriving at a measure o f a system’s value. If all other considerations were equal, 
a system with high PEMA and OMA costs would be rated lower than a cheaper 
system This technique was used to insure that these important cost considerations 
were included in the model, but not on the cost axis of the TVj versus cost curves, 
since the model is designed primarily for investigating only the RDTE budget. In 
addressing the problem of which, if any, systems could be dropped, the appropriate 
measure would be the total value (measured independent of all cost) per total 
system’s cost dollar. 
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See Wcingartncr (ref. 9| for a survey of these models. 
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Keeping in mind these considerations, CARDBOMB still might provide insight 
into the decision problem of which systems might be dropped. Using the total 
life expected value for the i** 1 system determined using algorithms 1 and 2, divide 
this by the total expected system’s cost. That is, for each system i, calculate 

TV® 

D: *=- — - 

*i + yi + I Pt z i 
t 

where 

0 

TVj ~ total life expected value at the optimal funding level. 

Xj = total RDTE cost at the optimal landing level 
yj = total PEMA cost. 

2 j = annual OMA cost. 

p< = as before, the probability the system is operational in year t. 

Ordering the set of D i in increasing sequence, while certainly not establishing a 
priority list for dropping systems, could assist the Army by focusing attention on 
which development programs might be more carefully analyzed. The first system 
in the list would be the one with the lowest average value, or value per total dollar. 

F. ANALYZING THE IMPACT ON THE TOTAL PROGRAM OF ADDING 
A NEW SYSTEM 

By the time a new system has finished the concept development stage, 
considerable cost-effectiveness analysis has been conducted on alternative approaches 
to meeting the threat for which it is designed. Besides these analyses a necessary 
input to the DCP is the impact the introduction of this system will have on other 
Army programs, i.e., what must be given up to get it. 
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As was the case in the question of which systems might be dropped, CARDBOM3 
is limited in assisting with this problem for much the same reasons — the inter¬ 
dependence of the value parameters and the treatment of PEMA and OMA costs as 
negative value rather than independent variables. However insight to this problem still 
might be gained in the following way. 

As was done for systems already in the program, survey category managers to 
determine 

Vi 

V' 

a measure of the value of this new i^ system on the same scale as all the other 
systems.^ Similarly, gather data for four alternative funding levels to arrive at 
a TVj versus RDTE cost curve. Include this new curve in the program and, using 
Algorithm 2 and the same total budget constraint, solve for an optimal solution. 

This new solution will determine two things. First, it will recommend at which of 
the five funding levels this system ought to be developed, and second, the impact 
on the systems whose funding levels had to be lowered determine a measure of 
what must be sacrificed in the RDTE program to get this system. That is, 
introducing System Z into the program might require reducing the design character¬ 
istics of System Y and stretching the operational readiness date of System W one 
year. Alternatively, the Army might decide to maintain the design characteristics 
of System Y, raire the RDTE budget constraint and make up the difference in some 
other budget program. In any event CARDbOMB has assisted by identifying in 
terms other than dollars, what must be given up to get System Z. 

way of avoiding the interdependence of value problem in s case is to 
survey to determine new relative value parameters for all the sy-.U-: ,n this 
category when the new system is considered part of this category. 
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Of course these measures of what must be given up only consider the RDTE 
budget. Lacking an optimization model which encompasses the RDTE, PEMA and 
OMA budgets, we must resort to the average value methods of the previous section. 
Therefore, besides conducting the analysis just discussed, it might also be appropriate 
to determine where the Dj for this system, the total value per total systems dollar, 
fits into the sequence of Dj discussed in the previous section. If it turned out to 
be high on the list, meaning it has one of the lower average values in the program, 
this could imply that it might not be worth the cost and should be studied further. 

G. BUDGETING PROJECTS WHICH ARE NOT COMPONENTS OF A SYSTEM 

DEVELOPMENT PLAN 

The discussion of the model has thus far ignored projects which are not 
related to a specific system’s development plan. In the classification scheme of 
this paper, those projects fall into Category 22, Research, and Category 21, 
Management and Support. We now turn our attention to theje two categories. 

The inputs required for an optimization model must answer two important 
questions; what is the value of the system, and how does this value vary with cost. 
Undoubtedly by the end of Chapter 3 the reader had developed an appreciation 
for the difficulty in providing quantitative answers to these questions. The problem 
is even more difficult when considering basic research and exploratory dcwNp: nent 
projects. Consider a hypothetical project designed for investigating military 
applications of a rotary engine. Unlike a system with a development plan which 
identifies expected performance characteristics, the threat it is designed to meet and 
other factors, just what might result from this project is highly uncertain. Equally 
uncertain is how a funding change will affect the results of the project. For these 
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reasons the model will treat category 22 projects differently than it has treated 
systems. 

Define twenty sub-categories of category 22 to correspond with the classification 
scheme for systems, i.e., sub-category 1 would be basic research projects associated 
with Air Mobility. Further define sub-category 21, Pure Research, consisting of 
all projects (excluding Management and Support) not otherwise classified. Using 
the methods of Chapter 2 for determining vame parameters of systems, obtain a 
parameter for each of the Category 22 projects. That is, for each of the projects 
within the j th sub-category, obtain a consensus measure, 


of the relative value of what is e xpected to result from the projects. Then use 
these with either the primary or secondary methods of Chapter 2 for determining 
the value of any sub-category project relative to the value of the Category 22 
numeraire, 



Ideally we would now like to know how this value varies with funds so that 
measures of marginal value could be obtained. Unfortunately basic research projects 
do not lend themselves to this measurement as well as system do. While an increase 
in funds might result in completing the project sooner, in general this advantage 
would be impossible to measure in terms of changes in value. However, under the 
assumption that for small funding changes near base case funding, the ratio of the 

For a more detailed discussion of considerations pertinent to modeling basic 
research, see Quinn (ref. 10) 
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marginal values of two projects are directly proportional to the ratio of their 
average values, we can develop a method for investigating alternative basic research 
budgets. 

We must first determine a method for measuring the value of the i^ project®® 
per total RDTE dollar, Xj being spent on it, defined as 



the average value. Recall that on an interval scale of measurement, the particular 
value assigned to the numeraire has no significance. Arbitrarily let Vq = Xq. We 
then have 


V- V- V V x 

_ I _ _1 . _fL- _i IS. 

^ v. V x* V x* 

A l v q q A l 


(9) 


Note that on this arbitrarily selected scale, the value per dollar, or average value, for 
the numeraire project reduces to Aq = 1. Another project which ’ ad only half the 
value cf the numeraire but was only one-tenth as expensive would have Aj “ 5. 

Before turning to the development of a decision rule, let us examine the 
implications of the assumption stated above. Figure 10 shows two basic research 




Figure 10. IMPLICATIONS OF THE AVERAGE VALUE ASSUMPTION 
®®The sub-category subscripts j are now dropped for simplicity oi station. 
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projects each funded at a total RDTE cost of xj = x^. Their values at this base funding 
were measured at 1 and V4 respectively. Their 1 average values, or value per dollar, are 
then measured z:> the slope of the dashed line from the origin. The solid curves 
are hypothetical value curves which, as has been mentioned, are not known. Since 
they are unknown, so is the marginal value at this funding level. If they were known 
we would want to increase the one with the greatest marginal value (or decrease 
the one with the smallest). Figure 10 shows that adding a dollar to the budget of 
project k would cause a greater increase in value than adding it to i, even though 
project i has the greater average value. This discussion is added to demonstrate how 
easily the assumption can be violated; the ratio of average values are not always 
proportional to the ratio of marginal values. However, lacking information on 
the curve itself, it might be further assumed that the base case funding is near the 
point where value is “falling off”, as is the case in the left figure. In other words 
the project has reached the point where hiring a few more scier .^ts just isn’t going 
to add that much. With this further assumption, we not continue with an 
algorithm for altering the Category 22 budget. 

We wish to insure tnat if the budget gets increased, the projects with higher 
average values receive proportionally more of the increase than those with lower 
average values. Conversely, if the budget is reduced, die lowest average value 
projects get reduced the most. These “decision rules” are depicted in Figures 11 
and 12, where 

A' =* max Aj and A" = min Aj 

i i 

Suppose a user of the model wishes to investigate the Categc-ry 92 buaget 
program that would result from using this method for a tcial const) int of D million 
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Figure 11. COMPUTING A PERCENTAGE INCREASE FOR 
BASIC RESEARCH PROJHCTS 



decrease 

Figure 12. COMPUTING A PERCENTAGE DECREASE FOR 
BASIC RESEARCH PROJECTS 


dollars, less than the base case program total constraint of E million. The following 
formula calculates the percentage reduction to each project that will be required to 
meet this new constraint. 


(A' - Aj) (E - D) 
‘ Z (A' - A,) C Wt „ 


( 10 ) 
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To illustrate the use of this formula, consider a three project basic research 
program with average value and FY 73 base funding costs as follows. 

Aj ■ A' * 5 C a - 10 

A 2 * 3 C 2 ■ 20 

A 3 * A" * 1 Cg -40 

Note that the total base case funding, E * 10 + 20 + 40 ■ 70. Suppose the user 
wished to investigate a constraint of D ■ 60. Equation 10 would result in the 
required percentage reduction for each project as shown. 


_ (5-5) (70-60) _ _0 _ 

(5-5) (10) + (5-3) (20) + (5-1) (40) “ 200~~ 


(5-3) (70-60) 1 

e --= — =10 percent 

^ onn in r 



It should be apparent that no optimization is involved in this method for altering 
the budgets of Category 22 projects. What is reflected in the solution, however, is 
that projects with high value per total dollar figures are increased proportionally 
more and decreased less when the total Basic Research budget Ls alt . d. Of course 
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the resulting solution would have to be carefully investigated to insure that the 
changes could be absorbed. 

To summarize, the decision rule for altering the budgets of individual Category 
22 projects is: If the budget constraint for Category 22 projects is greater (less) 
that the base case constraint by an amount D - E (E • D), then increase (decrease) 
each project the percentage of its base case funding as determined by Equation 11 
(Equation 10). 

Studying changes to the budgets of projects in Category 21, Management 
and Support, is not included in this paper. In the management, information system 
outlined in the next chapter the user can change the total for this category, but 
no method is developed here for investigating what changes might be considered 
for each project These changes would undoubtedly be made after a careful 
investigation of the organizational slack (fat) in each of tine headquarters elements. 

H. INVESTIGATING THE BUDGET LEVEL OF THE RDTE PROGRAM 

A characterization of an optimal solution for a given budget constraint (i.e., an 
optimal mix of systems’ budgets within the RDTE program) is that there is no 
other way that the funds can be distributed such that more total value will accrue 
to the program. This optima' (or efficient) mix implies nothing about what the 
optimal level, or total RDTE budget constraint should be. In this section a method 
is developed for investigating this question. 

Suppose that over the past several years, the RDTE budget has averaged about 
nine percent of the total Army budget and that under current DOD fiscal guidance, 
this nine percent represents 1.7 billion dollars. Suppose further that it has tentatively 
been decided to fund Categories 21 and 22 at 0.7 billion. The quev on is then. 
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should the remaining budget for systems be one billion, a little more or a little less? 
Insight into this question might be gained in the following way. 

Using Algorithm 2, find the approximation to the program of optimal mix for 
RDTE budget constraints ranging from 0.9 billion to 1.1 billion, in increments of 
10 million. For each of these 20 budget constraints, calculate 
n 

Z TV, 

i=l 

when each of the n systems are funded at optimality. Plotting these against the 
20 budget constraint points might lead to a curve similar to Figure 13. If such a 



Figure 13. TOTAL PROGRAM VALUE VERSUS TOTAL COST CURVE 

cur/e were to result, the implication is that increasing the budget from 0.9 to 
1 billion significantly increases the total program value, while a relatively smaller 
increase is achieved when going from 1 to 1.1 billion. 

It is impossible to predict without data what the shape of this curve might 
be. It will be non-decreasing, however, since it results from the addition of 
systems’ value curves which are non-decreasing. Whatever its exact shape turns out 
to be will determine how much insight into the problem this exercis' might provide. 
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For example, if a generally linear curve were to result, or c.ie where no discernable 
change in the rate of increase was apparent, relatively little insight would be 
provided in comparison with that implied by the Figure 13 curve. In fact a 
linear curve would provide no guidance in selecting a “proper” budget level, since 
it reflects no decrease at all in the rate at which total program value increases ts 
funds are increased. 
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V. IMPLEMENTING THE MODEL AS A 


MANAGEMENT INFORMATION SYSTEM 

Although the solution algorithms for this model are relatively simple, they 
require a great deal of data manipulation. The repetitious nature of these calculations 
lend themselves to computerization, and the model itself to a computer assisted 
management information system (MIS). In this chapter an MIS which could implement 
CARDBOMB is outlined; one which seems to be suitable for incorporation into 
the DEAN MACHINE system. The specific desires of the principal users would 
undoubtedly result in an MIS quite different than the one proposed here. This 
chapter is designed more to point out the potential uses of the model in assisting 
with the RDTE budget planning problem, and the relative simplicity of the computer 
programs required. It might also prove to be a usetul starting point in the develop¬ 
ment of a working MIS. 

A. THE BACK-UP SEGMENT OF THE MANAGEMENT INFORMATION SYSTEM 

Unlike pure costing models which, over time, tend to become “believeable” 
to users when their time saving capabilities and accuracy have been demonstrated, 
an optimization model of this type which incorporates many factors of a subjective 
nature and manipulates them in an almost “magical” fashion has to be clearly 
understood to be useful. The amount of insight into a decision problem that a 
particular solution might provide is directly proportional to how well the user 
understands the model. Consequently an MIS must consist of more than just a 

oq 

For a discussion of some of the pitfalls of designing an MIS, e Ackoff 
[ref. 111. 
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method for allowing a user to interface with a computer. A users’ manual which 


clearly states the assumptions, approximations, methods, capabilities and limitations 
of the model is an integral part of the MIS. Equally important is a document to 
which the user can refer when studying the parameters of systems he might wish 
to check while investigating a specific solution the model recommends. Although 
OCRD produces a Project Listing of the Program 6 base case, it is organized in the 
format of the budget and recognizing which projects belong to which systems is 
sometimes a difficult task. Additionally, only five years of cost information is 
available in this document. The reports of the Military Procurement Priority 
Review Board have systems’ priorities and costs, but generally not broken down by 
projects. The DCP’s for each system have a great amount of information for the 
base case, but this model also requires four alternatives to the base. Consequently, 
should CARD BOMB be implemented, it is recommended that a RDTE System’s 
Data Report be prepared and kept current. Besides being of "reat assistance 
to users of the model, it could prove to be an excellent planning document. A 
great deal of cost and effectiveness information on each system would be centralized, 
visible, and readily available for five alternative development plans. Such a document 
and the model itself, would no doubt be most efficiently maintained somewhere in 
the planning staff of OCRD, although its potential users might be personnel from 
the offices of the Director of the Army Budget, the Budget Review Committee, and 
the Assistant Vice Chief of Staff, in addition to OCRD. 

B. OPTIONAL USER INPUTS 

It is envisioned that this MIS wouid operate in a manner similar to the DEAN 
MACHINE system; i.e., the user-computer interface would be through, a cathode ray 
tube (CRT) console. The user types in simple instructions and solutions are displayed 


76 



on a television screen before him. Some of these instructions are presented in this 
section. 

Since the model is designed to handle cltemative methods for altering the RDTE 
budget, (e.g., alter the budget constraint and do not consider slipping, fix the 
constraint and consider slipping, alter the time preference strategy, etc.) the 
computer program segment of the MIS must allow for the input of thtae options 
if it is to be efficient in terms of running time and storage space. Consequently, 
a user should have the capability of inputing any on' the following options. 

1. Select a time preference strategy The most time consuming operation the 
program might be required to accomplish would be the construction of 
the TVj versus cost curves, calculation of .he marginal values (slopes), and 
the ordering of these slopes in decreasing sequence. It might therefore be 
desireable to have in storage, pre-constructed budget incrementing matrices 
for alternative strategies. A great deal of computer time could be saved 
this way. Therefore the foliowing time preference strategy options might 
be made available. 

Option 1, a Short Range (a, b, c) Strategy, where the parameters are those 
determined “best” in the test of the model against the base case (Chapter 4, 
Section C). 

Option 2, Neutral (1, 1, 1) Strategy. 

Option 3, Long Range (c, b, a) Strategy, using the same parameters as 
Option 1, but in reverse order 

Option 4, User selects his own parameters. As has been stated, the analyst 
should not impose a strategy on the decision maker. Option 4 allows the 
user to input his own time preference parameters, but he should be made 
aware of the fact that the solution will take much longer to obtain. 

2. Change any parameters? As familiarity with the model grows, the user 
may decide that he is not in agreement with some of the relative value, 
readiness dete distributions or other parameters on file. Alternatively 
he may wish to test the sensitivity of a particular solution to some of 
the value, cost or other parameters. Here he is given the option to 
temporarily change any of the parameters he wishes. This could be 
accomplished by typing a code number (defined in Section C) to identify 
the parameters and the new value he desires to use. 

3. Should the model consider slipping? A simple yes-no ansv. to this question 
would be sufficient to insure that the proper subroutines were used. 
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4. What fiscal year is being investigated? To answer this question the user 
types in the year. Although the development of the model dealt primarily 
with the.first year (F¥ 73), any budget year can be investigated. Suppose 
for a particular FY 73 constraint the optimal solution resulted in a total 
FY 75 cost that was considered unreasonable. A constraint could then 
be imposed on FY 75 and the funding levels of systems be reduced until 
this new constraint is met. 

6. Select the total RDTE budget constraint. The user would answer this 
question by typing in the dollar constraint he wishes to investigate. 

6. Select the constraint for basic research. If this were the same as for the 
base case, the model would by-pass the subroutine designed to alter the 
Category 22 budget. If it v/ere different, the model would calculate a 
new Category 22 budget according to the decision rules discussed in 
Chapter 4, Section G. 

7. Select the constraint for management and support. The model does 
nothing with this other than to subtract if from the total constraint to 
determine how much is left for the remaining systems. Should this 
constraint be less than the base case, the difference would represent the 
user’s personal judgement of the “organizational slack” that can be 
absorbed, 

8. Select the output options. At this point the user informs the computer 
program what he desires to see in the way of output. Discussion of what 
these options might b * is deferred to Section D of this chapter. 


C. ORGANIZATION OF THE DATA INPUTS 

A large number of data inputs are required for the use of CARDBOMB. How 

these are organized can have a significant effect on the efficiency of the computer 

program. Appendix B outlines the computer program and subroutines that would 

be needed for implementing the MIS in the fashion proposed in this chapter. As 

ment’oned in the discussion on inputing strategy motions, it has incorporated 

a trade-off of data storage for reduced running time. The following files would 

be required for operat : ng the MIS in the manner proposed in Appendix B. 

File 1, Budget Incrementing Matrix (BIM) for strategy option 1. The format 
for this pre-constructed matrix might be as was shown in Figure 7, page 

File 2, BIM for strategy option 2. Same format. 

File 3, BIM for strategy option 3. Same format. 
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File 4, Systems’ Parameter File. In order to use strategy option 4 or to change 
some of the parameters, a file consisting of the inputs required for constructing 
TVj versus cost curves is needed. Of course, this file is also necessary for 
constructing Files 1, 2 and 3. The format for this file might be as shown in 
Figure 14. 


[ — 

-f=2-][ - 

Total 

Total 

RDTE 

t 0 ^ RDTE 

l o Vn V *o t'm *p a a a Cost 

Cost V 


System 1 
2 
3 


Figure 14. FORMAT FOR FILE 4 

Note that a code (ij) would be sufficient to identify the parameter that a 
user might, wish to change. 

File 5, Basic Research Projects. The mathematical operations used in investigating 
alternative Category 22 budgets are different than those for systems. Grouping 
these projects separately allows for the separate investigation of alternative basic 
research budgets. The format for File 5 could take the form of that shown in 
Figure 15. 


Total 

RDTE 

73 74 75 .V Cost A 

Project Ij 
2 
3 

n 

Figure 15. FORMAT FOR FILE 5 
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File 6. Yearly Budget Schedule. It may have been noted in the formats 
recommended for the other five files that neither annual RDTE nor total 
PEMA and OMA costs for systems were included. This, of course, could 
easily have been done. However since the user is only investigating one year 
at a time, collecting this data in a separate file can assist in saving core storage 
space. Figure 16 shows the format this file might take. Note that if the user 
were investigating the FY 73 budget, only the columns pertaining to that 
year would need to be called into core memory. 


System 


Total Annual 
PEMA OMA 

1 

2 

3 

4 


[ -f=2-] [-f 3 . . . 

RDTE RDTE RDTE KDTE 

73 74 . 73 74 


Figure 16. FORMAT FOR FILE 6 

File 7, Financial Data File. Like the DEAN MACHINE, this model might be 
called upon to act as a “report generator”, transforming cost data from one 
budget format to another. None of the files outlined thus far i-ontain inform¬ 
ation on the costs (or identity) of the projects which compose the systems. 
This information could be included in Filt 7. Although Appendix B does 
not outline routines for incorporation of a report generation capability, this 
feature could be made a part of CARDBOMB. 


D. RELATINC COMPUTER OUTPUTS TO INFORMATION REQUIREMENTS 
There are a number of capabilities of CARDBOMB that a user might wish to 
employ. Allowing him to select an output option insures that the proper subroutines 
are called and the particular information he is seeking is displayed. How much 
information is required at a given time would direct whether it won: l>e best 
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displayed on the CRT or on a computer print-out. The following list of output 
options is not intended to be exhaustive. It might further demonstrate how the MIS 
could be used, however. 

1. CRT display of changes from the base case. Normally a user would wish 
to focus his attention on the changes that resulted from his inputing a 
new parameter or budget constraint. This could be accomplished by a 
visual display of the new versus base funding schedules of the system or 
basic research projects which changed. 

2. CRT display of changes from the last solution. It is envisioned that the 
model would be used in an iterative fashion, i.e., once a solution is generated 
with a certain set of parameters, the user might wish employ another 
option to see how it affects this solution. 

3. CRT Summary Display. The new and base funding schedules aggregated for 
each of the 22 categories might also be of interest. 

4. CRT display of the 12 rows of the Budget Incrementing Matrix bracketing 
the total budget constraint. For a particular budget constraint the model 
recommends changes based on this matrix. The user might also wish to 
see what other changes would be recommended if the constraint were 
slightly greater or slightly less. Twelve rows are selected because of the 
12 line output capability of the CRT. 

5. Computer print-cut of a Budget Incrementing Matrix. The BIM is an 
ordered list of increases to the budgets of systems as the total budget 
constraint is increased. Although it is only an approximation to the 
optimal solution, it could be used as a readily available planning document 
for answering the “What do you think we should do if the budget is cut” 
type question. 

6. Print-out of the slip sequence list. 

7. Print-out of the drop sequence list. 

8. Print-out of detailed funding schedules of all systems at optimality. 

9. Print-out of the complete RDTE budget in alternative formats. 
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VI. SUMMARY 


Throughout this paper an attempt has been made to point out both the capabil¬ 
ities and limitations of the methods used in the model. CARDBOMB is not meant 
to be either a “cure-all” for the RDTE budgeting problem or the final answer for 
the contribution analytical models might provide. What the model can do is 
provide logicul consistency between information gathered in the relatively “quiet” 
planning phase and the budget decisions made during the “crunch” period. Even 
during the “crunch” it is not envisioned that the solutions generated by the model 
should be arbitrarily accepted. However the budget feasible changes it recommends 
should serve to narrow the focus for a more careful investigation, one which might 
be well served by the comprehensive data bank. 

A. CAPABILITIES AND LIMITATIONS REVIEWED 

CARDBOMB has addressed individually the entire range of alternatives that 
can be employed for altering the RDTE budget. These were; increase or decrease 
the budgets of systems or projects, slip a system one or more years, add or drop 
a system or project from the program, and alter the RDTE budget total. Various 
analytical methods were used as noted below. 

Based on subjective value judgements of category experts and cost data supplied 
by program managers, value versus cost curves were determined for each system. 

Then for a particular budget constraint, an algorithm was used to generate an approxi¬ 
mation to the optimal RDTE budget. In this wav trade-offs of value for coct were 
accomplished for each system, but considered as a part of the total -ogram, not 



as an individual system. By this approximation method increases or decreases to 
the budgets of systems are recommended. Basic research and exploratory development 
projects are recommended for reduction or increase based on a decision rule which 
insures that percentage increases (decreases) are applied to the budget of projects 
which are directly (inversely) proportional to their average values. 

Dropping a system or project is also handled on an average value criteria. 

Because the model treats value interdependently and PEMA and OMA costs as negative 
value, its usefulness for assisting with this decision is limited. However it could be 
used to focus attention for further investigation on those systems with comparatively 
low average values. 

When deciding whether or not a proposed system’s development plan should 
be undertaken, the model can assist by identifying what needs to be given up in 
the HDTE program in terms of reduced funding levels for other systems. It can also 
help to identify the optimal funding level, i.e., the trade-off of cost for potential 
performance. Additionally, by comparing its average value with that of other systems, 
insight might be gained on whether or not the development plan should be undertaken. 

The decision on whether or not to slip a system is handled by determining a 
yearly loss in value that would result, and recommending those systems for slipping 
whose loss in value is less than the gain that would be achieved by raising the 
funding levels of other systems. 

It is hoped that the model could add insight into the “proper” total budget 
constraint by plotting a total value versus total budget curve in a budget interval 
around the historical |>ercentage of the total Army budget. As mentioned ir 
Chapter 4, the usefulness of this method is dependent on the shape of the resulting 
curve. 
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B. EXTENSIONS OF THE MODEL 


Some of the limitations discussed above are limitations only to the extent that 
they were beyond the scope of this paper. The interdependence of value of the 
systems which cause the model to be limited in its application to the add and drop 
decisions have been discussed at length by Weingartner [ref. 9], and theoretical 
models in the general category of project selection are available which could be 
adapted to CARDBOMB. The treatment of costs, however, would have to be 
handled differently, i.e., PEMA and OMA uncouf’ed from the measurement of 
value. This would suggest that perhaps the real problem is not the optimization 
of the RDTE budget but rather the optimization of the RDTE, PEMA and OMA 
budgets combined. Techniques similar to those used in this paper could undoubtedly 
be applied in most cases to this vastly expanded model. 

The average value decision ruie for basic research projects, though intuitively- 
appealing since it says to add the most money to those which are expected to 
return the most value per dollar, is not theoretically sound. The appropriate 
measurement for incremental changes in budgets is marginal value, not average 
value. A more thorough examination of Category 22 projects might suggest a 
more appropriate structure for modeling Basic Research. 

Of course a very important extension of the model would oe to implement it 
and see if it is useful in assisting in the RDTE budgeting problem. This would be 
a sizeable task th-** would no doubt be best accomplished in stages. For example 
very little would need to be changed in the computer programs for the final model 
if they were first written to accommodate the systems from only one category. 
Perhaps Category 5, STANOS, might be a suitable starting point since all these 
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systems come under a single program manager. Since the data manipulations for 
these few systems would not be excessive, the model could be tested without the use 
of the computer. 
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APPENDIX A 

MATHEMATICAL CONCEPTS 


This appendix is an extension of some of the concepts discussed in the main body 
of the paper. Details concerning these concepts were purposely omitted from the body 
of the text so that it would be more easily read by individuals not thoroughly 
familiar with the jargon of Operations Research. For those readers with an OR 
background, this appendix should assist in clarifying two important questions; what 
is embodied in the value parameters, and what is the formulation of the optimization 
program. 

The concept of relative value as used in this paper is similar to that found in 
Fishbum [lef. 12]. Let V be a value function which maps a real number V(s ; :) to 
a system sy from the set of systems Sj in the category. The survey respondent 
accomplishes this mapping considering 

1. Sj 

2. Njj the set of all perceived needs for systems in the category, where 

Nj is a function of, 

3. E; the environment — his perception of the enemy threat, the range 

of possible scenarios, etc. 

That is: 

V(Sy) - V(«y; Sj, Nj(E)] 

With the usual convention that if sy is considered more valuable than sjy then 
V(sy) > V(sjy), the optimization methods used require that V be unique up to 
an increasing linear transformation. In other words, the results of the survey on 
the zero-ten scale must be considered to have established an interval ‘-ale of 
measurement, meaning that shifting the origin of the scale or multiplying it by a 
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positive constant will leave the relative lengths of the intervals between V(sy) 
unchanged. For example: 

v («ij) - V(s kJ ) aV(sy) + b - [aV(s kJ ) + b] 

V <*lj> * V (*mj) ’ aV (lj) + b • t aV < 8 mj) + b] f0ra> ° 

This requirement for an interval scale of measurement could be tested by surveying 
again to confirm that the zero-ten scale produces the same relative interval results 
as, for example, a zero-one hundred scale. 

When the primary method (see page f) is used to relate the value of each 
system to that of a program numeraire, the high-level DA managers are asked to 
use the “ranking-rating” method for each of the category numeraires. Thus they 
are being asked to map real numbers to the values of these systems considering: 

1. Sj, .... S 2 q; the sets of systems in each category. 

2. S'; the set of numeraire systems. 

3. Nj, . . . , N 2 q; the set of needs in each category. 

4. E; the environment 
That is: 

v ( s qj) " V[s qJ ; S lf . . . , S 20 , S'. NjfE), . . . , N 20 (E)1 

Using the primary method for relating the value parameters of all the systems requires 
that the mapping V{s q j) should also result in an interval scale of measurement. The 
use of equation 2, page 29, is equivalent to finding “a” in the linear transformation, 
i.e., determining the constant 

Si 

v q,i 

which compresses or stretches the value scale of the category to • ake it 
commensurate with the scale of the program numeraire. 
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The secondary method, page 30, also seeks to determine 



but in a different manner. To use equation 4, page 32, 



must be determined by survey. The high-level managers must consider: 

1 . Nj_,.... N20 

2. E 

The value function mapping is then: 

V(Nj) = V[N j; Nj(E),. .., N 20 (E)] 

All the value measurement methods discussed in the text also require the additivity 
assumption, i.e., 

V( s lj + s 2 j + - • • + Sy) * V( SlJ ) + V(s 2 j) + . . . + V( S y) for i*l, . . ., nj 

For this assumption to hold, systems must be carefully defined. For example, a 
weapon and its platform must be one system. The assumption could be tested 
(with great effort) with surveys designed along the lines of the Churchman-Ackoff 
method for obtaining measures of relative value (see Burington [ref. 6, p. 26]). In 
brief, this would require that each pair of systems, then each triplet, etc., be tested 
by survey to insure that the additivity assumption holds. 

Without any testing of the parameters obtained by the “ranking-rating” method, 
there is no way of insuring that consistent judgement has been applied in formulating 
the value function, V. In other words it must be presumed that the usual axioms 
for the existence of a real valued utility function held, i.e., that the reference 
relation is reflexive, symetric and transitive. (Sec, for example, Fbi. .rn [ref. 12, 
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p. 167]). The testing required to insure that the axioms hold would result in a very 
sizeable surveying effort, one which is felt by the author to be an unreasonable 
undertaking in practice due to the time that might be allotted. Eckenrode [ref. 13] 
has shown in a limited test of the rating method that no statistically significant 
differences in results were obtained from this method and some of the more complex 
ones which do test for compliance with the axioms. Of course his study has only 
limited relevance to this particular problem. Consequently, the parameters which 
result from the relatively simple methods outlined in the body of the text should 
not be viewed as being perfectly correlated with some “actual” real valued 
utility function for the Department of the Army. The sensitivity of a solution 
to the relative value parameters used should always be checked. 

Algorithm 1, page 55, results from the formulation of the problem as a 
mathematical program in the following way. 

~n 0 0 0 

We wish to find the optimal funding level f u ** (f^, f 2 , • . . , f n ) which 
maximimizes the total value of the program and satisfies the budget constraint for 
the year being investigated. That is: 

Maximize: g( f) 

Subject to: h(f)<B t 

For the i 1 * 1 system, the problem has been formulated as: 

TVj(xj) » £ d t • at' p t (xj) • VjjXj : sy, Sj, Nj(E)] 
t 

Although a separable programming approach is not the only formulation that might 
be considered, it does result in the simplest solution methods. For g( f) to be a 
separable function it is necessary to assume that: 

v il*i . *jj. s j. N j( E )l * V^Xj) 
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that is, the value of the i** 1 system is independent of the values of all the others. As 
discussed in the text, it has been assumed that in the funding range [f = 2,. . . , f 6] 
this independence of systems' value holds. “Very often the choice between independ¬ 
ence or not is a choice between ‘divide and conquer with approximations’ or ‘don't 
divide and don’t conquer at all.’ ’’ Fishburn [ref. 12, p. 295]. However “don’t 
conquer at all’’ is not quite the case with this problem. As has been stated, 

Weingartner [ref. 9] discusses a number of models that take into account the 
interdependence of value. Some of them would be appropriate for the “add” and 
“drop” decision and could be adapted to CARDBOMB. 

With the assumption of independence of value, the program becomes: 

Maximize: g(f) “ Z TVj(xj) 

i 

Subject to: h( f) .< B t 

The budget constraint equation is logically separable into the year t budgets 

for each of the systems, i.e., h(f) =£ Cj t (fj) < B t , where Cj t (fj) is a discrete function 

i 

in f, the five finding levels. The final mathematical program is then: 

Maximize: £ TVj(xp 

i 

Subject to: £ Cj t (fj) S B t 

i 

fj * fj jj for k = 2, or 3 ... or 6 


Algorithm 1 accomplishes this optimization by first calculating the marginal values, 
Sj j t , of raising the funding level from f*ktof*k+l; i.e., 

TVftfi. k *l)*TV i (f iik ) 
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When the set of are ordered in decreasing sequence, iteratively increasing the 
funding levels according to this sequence insures that the increase has been added 
to the system with the highest gain in value per total RDTE dollar increase. However 
as was pointed out on page 54, this is not necessarily the optimal increase when a 
year t budget constraint is considered. Lacking information on these year t 
constraints for the out-years, the question becomes one of choosing a criteria; 
either maximize value considering the marginal value of total cost or the marginal 
value of year t cost. Of course Algorithm 1 uses the former criteria, but a user 
of the model might also wish to investigate a solution resulting from the latter. The 
two solutions would be identical if the percentage differences in total cost between 
each funding level were the same as the percentage differences in year t costs for 
all systems. In this case the sequence of Sj ^ would be the same as a sequence 
resulting from using C ( f+i t' ^i f t ’ n tJie denominator of the S: ^ formula. 

A number of nonlinear programming search techniques might, be applicable 
to finding the global optimum for a particular year t budget constraint. Unfortunately 
until such time as data might be collected and the shape of the TV- versus year t 
cost curves determined, it is impossible to predict how divergent the two solutions 
would be, or to recommend a particular solution algorithm. Of course if these curves 
were also cone ve in the region of positive value, using a sequence of marginal values 
of year t cost in Algorithm 1 would be a simple method. 
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APPENDIX B 
CONCEPT FLOW CHART 


This appendix is added to demonstrate the reasonably simple nature of the 
computer program that would be required to implement CARDBOMB. It is designed 
to give the reader a better understand of the MIS outlined in Chapter 5, not to 
present a detailed programming flow chart. Readers with some familiarity with 
programming may note that the computer operations required are neither complex 
or excessively time consuming. 
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Using (f,TV jf ) 
Calculate Slopes 


* 


Oider the Slopes 
Into a Budget 
Incremi r>ting 
Matrix 


Figure 18. STRATEGY OPTION 4 SUBROUTINE 




























Figure 19. PARAMETER CHANGE SUBROUTINE 
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Figure 20. BASIC RESEARCH SUBROUTINE 
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